remote
Senior Data Engineer - IKS Health
Data Engineer
Senior Data Engineer leading cloud‑based ELT/ETL pipeline design and maintenance, driving AI/ML infrastructure on AWS with Python, Spark, and Airflow to support high‑performance data science models.
About the role
Key Responsibilities
- Design, develop, and maintain scalable ELT/ETL pipelines using Python, SQL, and Apache Spark on AWS.
- Build and optimize data ingestion, transformation, and storage solutions to support AI/ML workloads.
- Collaborate with data scientists and ML engineers to deploy models into production, ensuring reliability and performance.
- Implement CI/CD pipelines, monitoring, and alerting for data workflows using Airflow and cloud native services.
- Document data architecture, data lineage, and best practices for data quality and governance.
Requirements
- 5+ years of data engineering experience with a strong focus on cloud data platforms.
- Proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with AWS services (S3, Redshift, Glue, EMR, Lambda).
- Experience building and maintaining Airflow DAGs and CI/CD for data pipelines.
- Strong understanding of ML Ops principles and model deployment pipelines.
Skills
pythonsqlapache sparkawsairflow