onsite
Senior Data Engineer - Assistant Vice President
Data Engineer
Lead enterprise data engineering initiatives, architecting scalable pipelines with Airflow, Iceberg, Kafka, Spark, and Delta Lake, while driving data quality, governance, and performance at a senior leadership level.
About the role
Key Responsibilities
- Design, build, and maintain end‑to‑end data pipelines using Apache Airflow, Spark, and Kafka to ingest, transform, and load large volumes of structured and unstructured data.
- Implement and manage Iceberg and Delta Lake tables, ensuring ACID compliance, schema evolution, and efficient query performance.
- Collaborate with data scientists, analysts, and product teams to define data requirements, optimize data models, and provide actionable insights.
- Lead data governance initiatives, including data quality monitoring, lineage tracking, and compliance with regulatory standards.
- Mentor and coach junior engineers, fostering a culture of continuous improvement and technical excellence.
Requirements
- 10+ years of experience in data engineering, with a proven track record in large‑scale, production‑grade data platforms.
- Deep expertise in Apache Airflow, Apache Iceberg, Apache Kafka, Apache Spark, and Delta Lake.
- Strong programming skills in Python and experience with SQL and distributed computing frameworks.
- Excellent problem‑solving abilities, communication skills, and a collaborative mindset.
- Experience leading cross‑functional teams and driving data strategy at an executive level.