onsite
Lead Data Engineer - TPXImpact Holdings Plc
Data Engineer
Lead the design, development, and optimization of scalable data pipelines, set engineering standards, and mentor teams while leveraging Python, Spark, Airflow, and AWS to deliver robust data solutions.
About the role
Key Responsibilities
- Architect, build, and maintain high‑performance data pipelines and ETL processes using Python, Apache Spark, and Apache Airflow.
- Define and enforce data engineering best practices, standards, and governance across multiple projects and client engagements.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable data solutions.
- Lead and mentor a team of data engineers, coordinating work, conducting code reviews, and fostering knowledge sharing.
- Identify opportunities to reuse existing data flows and components, improving efficiency and reducing time‑to‑value.
- Ensure data platform reliability, security, and cost‑effectiveness on cloud environments such as AWS.
Requirements
- 5+ years of hands‑on experience in data engineering, with a proven track record of designing and operating large‑scale data pipelines.
- Strong proficiency in Python, SQL, and big‑data processing frameworks (e.g., Apache Spark).
- Experience with workflow orchestration tools such as Apache Airflow or similar.
- Deep understanding of cloud services (AWS) including S3, Redshift, Glue, and IAM for data solutions.
- Excellent communication and leadership skills, with the ability to guide cross‑functional teams and influence technical direction.
Skills
pythonsqlapache sparkaws