Senior Data Engineer – AgileEngine
AgileEngine is an Inc.5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
WHY JOIN US
If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you
ABOUT THE ROLE
We are looking for a Senior Data Engineer to lead the replacement of a third-party digital ingestion tool with a custom-built ETL pipeline. This person will document existing logic, evaluate build-vs-buy options such as Fivetran, and build a standalone pipeline within an existing AWS environment. Strong SQL and DSP or programmatic advertising experience are central to this role.
WHAT YOU WILL DO
- Document current ingestion logic inside the legacy tool (TapClicks), including sources, endpoints, schemas, and transformation logic.
- Build the ETL pipeline to replace the legacy system, handling ingestion from the Amazon API,
other DSP platforms, and related digital data sources.
- Evaluate and decide between off-the-shelf solutions (e.g., Fivetran) and custom-built ELT systems alongside the Infrastructure Lead.
- Design the pipeline as a standalone, owned system to power client-facing dashboards.
- Provide versátil engineering support for application and backend work within the AWS environment when directed.
- Coordinate with data science teams to ensure ingestion output aligns with the attribution and scoring logic of downstream models.
- Work seamlessly within the existing AWS environment (ECS, S3, RDS) rather than standing up parallel infrastructure.
MUST HAVES
- 4+ years of strong SQL and large-scale data ingestion/pipeline experience (batch ETL, orchestration, schema design).
- Direct or adjacent experience with digital advertising/DSP data (e.g., Amazon DSP, Amazon Advertising API, or comparable programmatic platfo