onsite
Senior Data Engineer in Consulting - viadee Unternehmensberatung AG
Data Engineer
Lead end‑to‑end data engineering projects for consulting clients, building scalable pipelines on AWS, optimizing SQL and Spark workloads, and designing robust data models to support analytics and machine learning initiatives.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines using Python, SQL, and Apache Spark on AWS.
- Collaborate with data scientists and business stakeholders to translate requirements into scalable data solutions.
- Implement data quality checks, monitoring, and performance tuning for production workloads.
- Architect and document data models, lakehouse structures, and metadata management.
- Mentor junior engineers and conduct code reviews to ensure best practices.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python and SQL.
- Proven expertise in AWS services (S3, Redshift, Glue, EMR, Athena) and Spark.
- Hands‑on experience with ETL/ELT pipelines, data lakehouse concepts, and data modeling.
- Strong analytical skills and ability to troubleshoot complex data issues.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlawsapache spark