onsite
Data Engineer - Hampshire County Council
Data Engineer
Data Engineer role focused on designing, building, and maintaining scalable data pipelines and warehouses using Python, SQL, Azure and modern ETL tools to support children’s services analytics.
About the role
Key Responsibilities
- Design, develop, and optimise end‑to‑end data pipelines to ingest, transform, and load large volumes of structured and unstructured data.
- Build and maintain data warehouse solutions on Azure, ensuring high performance, reliability and security.
- Collaborate with data analysts, data scientists and business stakeholders to translate requirements into technical specifications.
- Implement and manage ETL processes using tools such as Azure Data Factory, Apache Spark and Docker‑based environments.
- Monitor data quality, perform root‑cause analysis of issues, and implement automated testing and validation frameworks.
Requirements
- Proven experience as a Data Engineer with strong Python and SQL programming skills.
- Hands‑on experience with Azure cloud services (e.g., Azure Synapse, Data Factory, Blob Storage).
- Solid understanding of data warehousing concepts, dimensional modelling and ETL best practices.
- Familiarity with big‑data processing frameworks such as Apache Spark and containerisation using Docker.
- Ability to work independently, communicate technical concepts clearly, and mentor junior team members.
Skills
pythonsqlazureapache sparkdocker