remote
Data Engineering Consultant - Optum
Software Engineer
Senior Data Engineering Consultant building scalable data pipelines using Python, Spark, and AWS services to deliver actionable insights for healthcare optimization.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load large volumes of healthcare data into cloud data warehouses.
- Leverage Python, SQL, and Apache Spark to build efficient ETL processes and perform data quality checks.
- Implement and manage orchestration workflows with Airflow, ensuring reliability and monitoring of data pipelines.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and translate them into technical solutions.
- Optimize performance and cost of data pipelines on AWS, utilizing services such as S3, Redshift, Glue, and EMR.
Requirements
- 5+ years of experience in data engineering within a healthcare or large-scale data environment.
- Strong proficiency in Python, SQL, and Spark for data processing.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, EMR) and workflow orchestration (Airflow).
- Solid understanding of data modeling, data warehousing concepts, and data governance best practices.
- Excellent problem‑solving skills and ability to communicate complex technical concepts to non‑technical stakeholders.
Skills
pythonsqlapache sparkawsairflow