onsite
Technical Data Engineering Specialist - fabplus GmbH
Software Engineer
Lead end‑to‑end data pipeline development, architecting scalable solutions on AWS, and optimizing ETL workflows with Python, SQL, Spark, and Airflow to deliver high‑quality data for analytics and machine learning.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines from diverse sources to cloud data warehouses, ensuring data quality and reliability.
- Implement ETL processes using Python, SQL, and Spark, optimizing performance for large‑scale datasets.
- Leverage AWS services (S3, Redshift, Glue, Lambda) to create scalable, cost‑effective data architectures.
- Develop and manage Airflow DAGs for automated workflow orchestration and monitoring.
- Collaborate with data scientists and product teams to understand data needs and deliver actionable insights.
- Document data models, pipeline logic, and best practices for maintainability and knowledge transfer.
Requirements
- Proven experience in data engineering, with strong Python and SQL skills.
- Hands‑on expertise with AWS data services and big‑data frameworks such as Spark.
- Solid understanding of ETL concepts, data modeling, and data quality principles.
- Experience with workflow orchestration tools, preferably Airflow.
- Excellent problem‑solving skills and a collaborative mindset.
Skills
pythonsqlawsapache sparkairflow