onsite
Director, P3, Data Engineer III
Data Engineer
Lead a high‑impact data engineering team, architecting scalable pipelines with Apache Airflow, Spark, and Dask on Azure, while ensuring robust data governance and performance across DB2 and cloud environments.
About the role
Key Responsibilities
- Design, implement, and maintain end‑to‑end data pipelines using Apache Airflow, Spark, and Dask on Azure.
- Lead a team of senior data engineers, providing mentorship, code reviews, and career development.
- Collaborate with data scientists and product owners to translate business requirements into scalable data solutions.
- Ensure data quality, security, and compliance across on‑prem DB2 and cloud data stores.
- Drive continuous improvement of data architecture, tooling, and best practices.
Requirements
- 10+ years of data engineering experience with a proven track record in large‑scale pipeline development.
- Expertise in Apache Airflow, Spark, Dask, Azure Data Services, and DB2.
- Strong leadership skills with experience managing and scaling engineering teams.
- Excellent problem‑solving, communication, and stakeholder management abilities.