onsite
Data Engineer - Hamburger Hochbahn AG
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and lakehouse architecture using Python, SQL, and AWS services, ensuring high data quality and performance for analytics and machine learning workloads.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines and lakehouse solutions on AWS, ensuring data availability and reliability for downstream analytics and ML teams.
- Implement ETL processes using Python, Spark, and SQL, optimizing for performance and scalability across large datasets.
- Collaborate with data scientists and business analysts to define data models, schemas, and metadata management strategies.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and processing efficiency.
- Document architecture, data flows, and best practices for future maintenance and onboarding.
Requirements
- Proven experience as a Data Engineer or similar role, with strong Python and SQL skills.
- Hands‑on experience with AWS services (S3, Glue, Redshift, Athena, EMR) and Spark-based data processing.
- Solid understanding of data modeling, ETL design patterns, and lakehouse concepts.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
- Strong communication skills in German and English.