onsite
Senior Data Engineer - Corewell Health
Data Engineer
Senior Data Engineer building and optimizing end‑to‑end data pipelines across on‑prem and cloud platforms, turning raw data into reliable, timely analytics assets for data scientists and business users.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, SQL, and Spark across on‑prem and cloud environments.
- Collaborate with architects, BI developers, analysts, and data scientists to translate business requirements into robust data models and workflows.
- Implement and manage orchestration with Airflow, ensuring reliable scheduling, monitoring, and alerting of data processes.
- Optimize data ingestion, transformation, and storage to improve performance, reduce latency, and lower costs.
- Document data lineage, quality metrics, and best practices for data governance and compliance.
Requirements
- 5+ years of experience as a data engineer or similar role.
- Proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with cloud data platforms (AWS, GCP, or Azure) and data warehousing solutions.
- Strong understanding of ETL/ELT patterns, data modeling, and performance tuning.
- Excellent problem‑solving skills and ability to work collaboratively in cross‑functional teams.
Skills
pythonsqlapache sparkairflow