onsite
Data Engineer - HUK-COBURG Versicherungsgruppe
Data Engineer
Design, build, and maintain scalable data pipelines and infrastructure using Python, SQL, and AWS services, ensuring high data quality and performance for analytics and machine learning workloads.
About the role
Key Responsibilities
- Develop and maintain robust ETL pipelines using Python, SQL, and Apache Spark to ingest, transform, and load data from diverse sources into data lakes and warehouses.
- Design and implement data models and schemas that support business intelligence, reporting, and advanced analytics.
- Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver high‑quality, reproducible datasets.
- Automate workflow orchestration with Airflow, ensuring reliability, monitoring, and alerting for production data pipelines.
- Optimize query performance and storage costs on AWS services such as Redshift, S3, and Glue.
- Document data architecture, pipeline logic, and best practices for maintainability and knowledge transfer.
Requirements
- 3+ years of experience as a data engineer or similar role in a data‑centric environment.
- Hands‑on experience with workflow orchestration tools such as Airflow or Prefect.
- Excellent problem‑solving skills, attention to detail, and ability to communicate complex technical concepts to non‑technical stakeholders.
Skills
pythonsqlawsapache sparkairflow