27 oct
|
Siriusbeach
|
Mendoza
27 oct
Siriusbeach
Mendoza
The Role: Builds performant data processing pipelines, manages relational databases, and constructs cloud-portable data lake environments using modern open storage formats and embedded analytical engines.
Responsibilities: Pipeline Design: Construct resilient data processing pipelines in Python using Medallion architecture principles (Bronze/Silver/Gold) to transform raw data into analytics-ready assets.
Open Storage Management: Standardize data storage using open formats including Parquet, Apache Iceberg, and Delta Lake to ensure cross-cloud compatibility.
Embedded Analytics: Leverage DuckDB integrated with Python for fast, lightweight, and performant localized data processing within ETL workflows.
Database Operations: Architect and optimize production PostgreSQL schemas while utilizing SQLite for embedded or lightweight localized storage needs.
Distributed Compute: Scale heavy ETL, data transformation, and analytics workloads using Databricks or Apache Spark
Requirements:
Advanced SQL proficiency with deep PostgreSQL expertise and working knowledge of SQLite.
Hands-on experience building robust analytical pipelines using Python and DuckDB.
Mastery of open table and storage formats (Parquet, Apache Iceberg, Delta Lake).
Proven track record designing data lakes using the Medallion data processing architecture.
Familiarity with the Databricks / PySpark ecosystem for distributed processing.
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you
ABOUT THE ROLE
We are looking for a Senior Data Engineer to replace a third-party data ingestion tool with an owned ETL pipeline.
MUST HAVES
- 4+ years of strong SQL and large-scale data ingestion/pipeline experience (batch ETL, orchestration, schema design).
- Working knowledge of AWS data services (RDS/Postgres, S3, ECS) sufficient to build within an existing cloud environment.
- Practical experience with Claude Code or similar tools, and agents or MCP in real work.
- Ability to contribute to application and backend work beyond pure ETL to support broader platform needs.
- Familiarity with, or ability to quickly evaluate, managed ELT tools (such as Fivetran) against custom builds.
- Ability to ramp into existing attribution and scoring logic so ingestion output aligns with downstream dashboard needs.
NICE TO HAVES
- Direct or adjacent experience with digital advertising / DSP data (e.g., Amazon DSP, Amazon Advertising API, or comparable programmatic platforms).
- Experience replacing legacy third-party data pipelines with owned, custom infrastructure.
- Familiarity with MLOps pipelines and how data feeds predictive models.
- Experience working alongside part-time technical advisory leads to validate architecture decisions.
WHAT YOU WILL DO
- Document current ingestion logic inside the legacy tool, including sources, endpoints, schemas, and transformation logic.
- Build the ETL pipeline to replace the legacy system, handling ingestion from the Amazon API, other DSP platforms, and related digital data sources.
- Evaluate and decide between off-the-shelf solutions (e.g., Fivetran) and owned ETL systems alongside the Infrastructure Lead.
- Design the pipeline as a standalone, owned system to power client-facing dashboards.
- Provide adaptable engineering support for application and backend work within the AWS environment when directed.
- Coordinate with data science teams to ensure ingestion output aligns with the attribution and scoring logic of downstream models.
- Work seamlessly within the existing AWS environment (ECS, S3, RDS) rather than standing up parallel infrastructure.
PERKS AND BENEFITS
- Professional growth: Mentorship, TechTalks, and personalized growth roadmaps.
- Competitive compensation: USD-based pay with education, fitness, and team activity budgets.
- Exciting projects: Modern solutions with Fortune 500 and top product companies.
- Flextime: Flexible schedule with remote and office options.
Job Description
AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOI
📌 Data Engineer (Mendoza)
🏢 Siriusbeach
📍 Mendoza