onsite
Group Manager - Data Engineering
Software Engineer
Lead a data engineering team to design, build, and maintain scalable data pipelines using Apache Airflow, Spark, and BigQuery while driving CI/CD practices in an Agile environment.
About the role
Key Responsibilities
- Lead and mentor a team of data engineers, fostering a collaborative, high‑performance culture.
- Architect, develop, and optimize end‑to‑end data pipelines with Apache Airflow and Apache Spark, ensuring reliability and scalability.
- Design data models and implement analytics solutions on Google BigQuery to support business intelligence and reporting needs.
- Establish and maintain CI/CD pipelines for data workflows, promoting automated testing, deployment, and monitoring.
- Drive Agile processes, including sprint planning, backlog grooming, and continuous improvement initiatives.
Requirements
- 5+ years of hands‑on experience in data engineering, with deep expertise in Apache Airflow, Apache Spark, and Google BigQuery.
- Proven track record of leading technical teams and delivering complex data projects on schedule.
- Strong knowledge of CI/CD tools and practices (e.g., Git, Jenkins, GitLab CI, Terraform).
- Experience working in Agile environments, comfortable with Scrum ceremonies and iterative development.
- Excellent problem‑solving, communication, and stakeholder management skills.
Skills
apache sparkcicdagile