onsite
Senior Data Engineer - LNER
Data Engineer
Senior Data Engineer driving data‑driven culture on a flagship rail route, building scalable pipelines, data lakes, and governance frameworks using Python, Spark, AWS, and Airflow.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines that ingest, transform, and serve data for analytics and operational use.
- Build and manage a cloud‑based data lake on AWS, ensuring data quality, lineage, and security.
- Implement data modeling and schema design to support business reporting and advanced analytics.
- Collaborate with data scientists, product owners, and stakeholders to translate business requirements into technical solutions.
- Automate workflows with Airflow, monitor performance, and troubleshoot issues across the data stack.
Requirements
- 5+ years of experience as a data engineer in a production environment.
- Proficiency in Python, SQL, and Apache Spark for data processing.
- Hands‑on experience with AWS services (S3, Glue, Redshift, EMR).
- Strong understanding of data modeling, ETL best practices, and data governance.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced, customer‑focused setting.
Skills
pythonsqlapache sparkawsairflow