onsite
Data Engineer - FERCHAU GmbH Niederlassung Osnabruck
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and infrastructure using Python, SQL, AWS, Spark, Docker, and Airflow to enable data-driven decision making.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines and ETL processes to ingest, transform, and load data from diverse sources into data warehouses and lakes.
- Implement and optimize data models, schemas, and indexing strategies for high-performance analytics on AWS services such as Redshift, S3, and Glue.
- Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver reliable, high-quality datasets.
- Automate workflow orchestration using Airflow, ensuring scheduled jobs run reliably and are monitored for failures.
- Containerize services with Docker and manage deployment pipelines for continuous integration and delivery.
- Monitor system performance, troubleshoot issues, and implement best practices for data quality, security, and compliance.
Requirements
- Proven experience as a Data Engineer or similar role, with strong proficiency in Python and SQL.
- Hands‑on experience with AWS data services (Redshift, S3, Glue, Athena) and big‑data frameworks (Apache Spark).
- Solid understanding of data modeling, ETL design, and performance tuning.
- Experience with workflow orchestration (Airflow) and containerization (Docker).
- Excellent problem‑solving skills, strong communication, and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache sparkdockerairflow