onsite
Software Engineer, Data Integration - aaru
Software Engineer
Build scalable data pipelines that ingest, transform, and deliver high‑quality data for AI agents, leveraging Python, SQL, ETL, and AWS services to support predictive intelligence at scale.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest raw data from diverse sources into the AI platform.
- Implement ETL processes using Python, SQL, and Apache Spark to transform and enrich data for downstream analytics.
- Optimize pipeline performance and reliability on AWS infrastructure (S3, Glue, Redshift, Lambda).
- Collaborate with data scientists and product teams to understand data requirements and deliver actionable insights.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and processing efficiency.
Requirements
- 3+ years of experience building data pipelines in a production environment.
- Experience with big‑data frameworks such as Apache Spark or similar.
- Excellent problem‑solving skills and a passion for clean, maintainable code.
Skills
pythonsqlawsapache spark