onsite
Senior Data Engineer - Kaufland e-commerce Services GmbH & Co. KG
Data Engineer
Lead end‑to‑end data pipeline development, architecting scalable solutions on AWS, and optimizing SQL and Spark workloads to deliver high‑quality data products for e‑commerce analytics.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines using Python, SQL, and Apache Spark on AWS services (Glue, EMR, Redshift).
- Collaborate with data scientists and product teams to translate business requirements into scalable data models and ETL processes.
- Implement data quality checks, monitoring, and alerting to ensure reliability and performance of data workflows.
- Optimize query performance and resource utilization across large datasets, applying best practices in partitioning, indexing, and caching.
- Mentor junior engineers, conduct code reviews, and promote data engineering best practices within the team.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python, SQL, and Spark.
- Proven expertise in AWS data services (Glue, EMR, Redshift, S3, Athena).
- Solid understanding of data modeling, ETL design, and performance tuning.
- Experience with CI/CD pipelines, version control (Git), and automated testing.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlawsapache spark