Data Engineer - Claims Recruitment Services Limited
Data Engineer
Data Engineer building scalable data pipelines in AWS, using Python, SQL and Spark to support data science and business intelligence for public sector clients.
About the role
Key Responsibilities
Design, develop and maintain end‑to‑end data pipelines on AWS (Glue, Redshift, S3) to ingest, transform and store large volumes of structured and semi‑structured data.
Implement robust ETL processes using Python and Spark, ensuring data quality, lineage and performance optimization.
Collaborate with data scientists and business analysts to translate analytical requirements into scalable data models and dashboards.
Monitor pipeline health, troubleshoot issues, and continuously improve reliability and efficiency.
Document architecture, code, and best practices for future maintainability.
Requirements
3+ years of experience in data engineering, preferably in a consulting or public sector environment.
Strong proficiency in Python, SQL and Apache Spark.
Hands‑on experience with AWS services (Glue, Redshift, S3, Lambda).
Solid understanding of data modeling, ETL design patterns and data warehousing concepts.
Excellent problem‑solving skills and ability to work collaboratively in cross‑functional teams.