onsite
Lead Data Engineer - TPXimpact
Data Engineer
Lead a data engineering team to design, build, and optimise scalable data pipelines using Python, Spark, Airflow, and AWS while establishing best practices and driving strategic data solutions for clients.
About the role
Key Responsibilities
- Architect, develop, and maintain high‑performance data pipelines and ETL processes across cloud and on‑prem environments.
- Define and enforce data engineering standards, coding guidelines, and best practices for the team.
- Mentor and lead engineers, fostering a collaborative culture focused on quality, scalability, and client impact.
- Identify opportunities to reuse existing data flows and improve data architecture for efficiency.
- Collaborate with analytics, data science, and business stakeholders to translate requirements into robust data solutions.
Requirements
- 5+ years of hands‑on experience building data pipelines with Python, SQL, and Spark.
- Proficiency in workflow orchestration tools such as Apache Airflow.
- Strong knowledge of cloud platforms, preferably AWS (S3, Redshift, Glue, Lambda).
- Experience designing data models and implementing data warehousing solutions.
- Proven leadership or lead‑engineer experience, with excellent communication and stakeholder‑management skills.
Skills
pythonsqlapache sparkaws