onsite
Senior Data Engineer - Oak Street Health, part of CVS Health
Data Engineer
Senior Data Engineer responsible for designing, building, and maintaining large‑scale data pipelines and ETL workflows using Python, Spark, Airflow, and AWS services to support analytics and business intelligence across the organization.
About the role
Key Responsibilities
- Design, develop, and optimize scalable data pipelines and ETL processes using Python, Spark, and Airflow.
- Collaborate with data scientists and business analysts to understand data requirements and deliver high‑quality data assets.
- Implement data modeling best practices and maintain metadata catalogs for enterprise data lake and warehouse environments.
- Ensure data quality, lineage, and governance through automated testing, monitoring, and documentation.
- Leverage AWS services (S3, Redshift, Glue, EMR) to support data ingestion, transformation, and storage at petabyte scale.
Requirements
- 5+ years of experience in data engineering with a strong background in Python and SQL.
- Proficiency in distributed processing frameworks such as Apache Spark and workflow orchestration tools like Airflow.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, EMR).
- Solid understanding of data modeling, ETL design patterns, and data governance principles.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlapache sparkairflowaws