onsite
Cloud Data Engineer - AWS and Databricks - ivv
Data Engineer
Join a leading insurer as a Cloud Data Engineer, building scalable data pipelines on AWS and Databricks. Leverage Python, Spark, and SQL to transform and analyze large datasets, driving data‑driven decisions across the organization.
About the role
Key Responsibilities
- Design, develop, and maintain data pipelines on AWS and Databricks to ingest, transform, and store large volumes of structured and unstructured data.
- Implement ETL processes using Python and Apache Spark, ensuring data quality, performance, and scalability.
- Collaborate with data scientists and business analysts to deliver actionable insights through data models and dashboards.
- Monitor and optimize pipeline performance, troubleshoot issues, and implement best practices for data governance.
- Document architecture, code, and processes for knowledge sharing and compliance.
Requirements
- Strong experience with AWS services (S3, Glue, Redshift, EMR) and Databricks.
- Proficiency in Python, SQL, and Apache Spark for data processing.
- Solid understanding of data modeling, ETL concepts, and data warehousing.
- Excellent problem‑solving skills and ability to work in a fast‑paced environment.
- Good communication skills in German and English.
Skills
awsdatabrickspythonapache sparksql