remote
Lead Data Engineer - Datatech Analytics
Data Engineer
Lead a high‑performing data engineering team, architecting scalable pipelines and standards for an AI SaaS platform using Python, Spark, SQL, AWS, and Airflow.
About the role
Key Responsibilities
- Architect, build, and maintain large‑scale data pipelines that support AI and analytics workloads.
- Define and enforce engineering standards, best practices, and governance across the data platform.
- Mentor and grow a team of data engineers, ensuring hands‑on involvement and continuous skill development.
- Collaborate with data scientists, product, and infrastructure teams to deliver reliable, high‑quality data assets.
- Drive performance tuning, cost optimisation, and reliability improvements on cloud‑based data services.
Requirements
- 5+ years of data engineering experience in a fast‑moving SaaS environment.
- Proficiency in Python, SQL, and Apache Spark for batch and streaming workloads.
- Hands‑on experience with AWS services (S3, Redshift, Glue, EMR) and orchestration tools like Airflow.
- Strong understanding of data modelling, ETL design, and data quality practices.
- Excellent communication skills and a proven track record of leading technical teams.
Skills
pythonsqlapache sparkawsairflow