onsite
Senior Data Engineer - Department for Work and Pensions
Data Engineer
Lead the design and implementation of cloud‑based data pipelines, leveraging Python, SQL, AWS, and modern data warehousing techniques to deliver high‑quality data products that drive insights for the Department for Work and Pensions.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using Python, SQL, and AWS services.
- Implement ETL processes and data transformation logic to support analytics and reporting.
- Collaborate with data scientists and business stakeholders to define data requirements and deliver actionable insights.
- Optimize data storage and retrieval in data warehouses, ensuring performance and cost efficiency.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and reliability.
Requirements
- Proven experience as a Data Engineer with strong Python and SQL skills.
- Hands‑on expertise with AWS data services (Redshift, S3, Glue, Athena).
- Experience building ETL workflows using Airflow or similar orchestration tools.
- Knowledge of big data technologies such as Apache Spark and data warehousing concepts.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache sparkairflow