onsite
Data Engineer - Data Exchange & Automation Hub - ALDI Einkauf SE & Co. OHG
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines that enable seamless data exchange and automation across the organization, leveraging Python, SQL, and cloud technologies.
About the role
Key Responsibilities
- Design, develop, and optimize robust ETL pipelines to ingest, transform, and load data from diverse sources into centralized data lakes and warehouses.
- Collaborate with data scientists, analysts, and business stakeholders to define data models, schemas, and governance standards that support analytics and reporting.
- Implement and maintain data quality checks, monitoring, and alerting to ensure high data integrity and availability.
- Leverage AWS services (S3, Glue, Redshift, Athena) and Apache Spark to process large-scale datasets efficiently.
- Automate workflow orchestration using Airflow, ensuring repeatable, scalable, and fault‑tolerant data processes.
- Document data pipelines, architecture decisions, and best practices for internal teams.
Requirements
- Proven experience as a Data Engineer or similar role, with strong proficiency in Python and SQL.
- Hands‑on experience building ETL pipelines and working with cloud data platforms, preferably AWS.
- Familiarity with big‑data processing frameworks such as Apache Spark and workflow orchestration tools like Airflow.
- Solid understanding of data modeling, data warehousing concepts, and data governance principles.
- Excellent problem‑solving skills and a collaborative mindset to work across cross‑functional teams.
Skills
pythonsqlawsapache sparkairflow