03 oct
|
Marvik
|
Argentina
This is a hands-on building role: you turn raw, messy fabrication data into the clean, well-modeled, AI-ready datasets that our AI/ML and analytics workloads run on
Responsibilities:
- Pipeline Development & Operation: Build and operate scalable ingestion, ELT/ETL, and orchestration pipelines (batch and real-time streaming) within Microsoft Fabric and cloud lakehouse environments.
- Real-Time Data Ingestion: Design and implement low-latency, real-time data ingestion flows to support live operational analytics and streaming workloads.
- Data Modeling & Layering: Implement layered (medallion-style: Bronze/Silver/Gold) architectures using PySpark/SQL with idempotent, backfillable, and incrementally loaded jobs.
- Data Quality & Governance: Apply deduplication, normalization, schema validation, and lineage tracking to ensure downstream data is high-quality, trustworthy, and audit-ready.
- AI & Analytics Readiness: Deliver feature-ready, curated datasets to support business intelligence, analytics, vector search, and AI/ML agentic workloads.
- Observability & Reliability:
Establish testing, monitoring, and pipeline observability (freshness, volume, schema drift) with clear alerting to resolve failures proactively.
- Tooling & AI Development: Utilize AI-assisted development tools (Claude Code, Copilot, Cursor) as a force multiplier for writing pipelines, query tuning, and data transformation scripts.
If you have:
- Experience: 5+ years of hands‑on data engineering experience building and operating production data pipelines at scale.
- Core Technical Stack: Strong proficiency in Python, SQL, and PySpark / Apache Spark, backed by solid software engineering fundamentals (Git, CI/CD, unit/integration testing).
- Real-Time Data Processing: Demonstrated hands‑on experience implementing real-time data ingestion and streaming pipelines (not limited to batch processing).
- Data Architecture & Modeling: Proven experience in end-to-end data modeling, schema design, and layered lak
📌 Senior Data Engineer (Argentina)
🏢 Marvik
📍 Argentina