onsite
Senior Data Engineer and Data Manager - Welsh Revenue Authority
Data Engineer
Lead the design, development and maintenance of enterprise data pipelines and warehouses, ensuring high quality, scalable solutions using Python, SQL, ETL tools and AWS services.
About the role
Key Responsibilities
- Architect and implement robust data pipelines that ingest, transform and load large volumes of data from diverse sources into a centralized data warehouse.
- Design and maintain data models, schemas and metadata to support analytics, reporting and decision‑making across the organization.
- Collaborate with data scientists, analysts and business stakeholders to understand requirements and deliver high‑quality data assets.
- Optimize performance of SQL queries and ETL processes, ensuring efficient use of cloud resources and cost control.
- Implement data governance, security and compliance practices, including data lineage, quality checks and audit trails.
Requirements
- Proven experience as a data engineer or similar role, with a strong background in Python, SQL and ETL development.
- Hands‑on expertise with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery) and cloud platforms (AWS).
- Solid understanding of data modeling, schema design and performance tuning.
- Experience with data governance, security and compliance frameworks.
- Excellent communication skills and ability to work collaboratively in a cross‑functional team.