remote
Data Engineer - The Telegraph
Data Engineer
Data Engineer driving scalable, reliable data platforms and real‑time streaming solutions using Python, Spark, Kafka and AWS, while collaborating with cross‑functional teams to shape architecture and optimize business processes.
About the role
Key Responsibilities
- Design, build and maintain scalable data pipelines and streaming architectures using Apache Spark and Kafka.
- Develop and optimize ETL processes in Python and SQL to support analytics and data science initiatives.
- Collaborate with Data Scientists, Analysts and Product teams to define data models and ensure data quality.
- Implement best practices for data governance, security and performance across the data platform.
- Monitor and troubleshoot production data flows, ensuring high availability and reliability.
Requirements
- Proven experience as a Data Engineer or similar role in a production environment.
- Strong proficiency in Python, SQL and experience with Spark and Kafka.
- Hands‑on knowledge of AWS services (S3, Redshift, EMR, Glue).
- Solid understanding of data modeling, ETL design and data warehousing concepts.
- Excellent problem‑solving skills and ability to work collaboratively in a cross‑functional team.
Skills
pythonsqlapache sparkaws