remote
Data Operations Engineer - Abaka AI
Systems Engineer
Lead the design, implementation, and maintenance of scalable data pipelines using Python, SQL, Airflow, and AWS services to support AI workloads and analytics.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL pipelines that ingest, transform, and load large volumes of structured and unstructured data into data lakes and warehouses.
- Collaborate with data scientists and product teams to understand data requirements and translate them into efficient, reusable data workflows.
- Implement and manage Airflow DAGs, ensuring reliability, scalability, and observability of data pipelines.
- Optimize SQL queries and data models for performance and cost-efficiency on AWS services such as Redshift, Athena, and S3.
- Monitor pipeline health, troubleshoot failures, and proactively address bottlenecks using monitoring tools and alerting systems.
Requirements
- 3+ years of experience in data engineering or related roles, with a strong focus on pipeline development.
- Proficiency in Python, SQL, and experience with Airflow or similar workflow orchestration tools.
- Hands‑on experience with AWS data services (S3, Redshift, Athena, Glue).
- Solid understanding of data modeling, schema design, and best practices for data quality and governance.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlairflowaws