onsite
Data Engineer - FERCHAU GmbH Niederlassung Koln
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and infrastructure using Python, SQL, Spark, and AWS services. Focus on data quality, automation, and collaboration with analytics teams.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines and ETL processes using Python, SQL, and Apache Spark.
- Implement and manage data storage solutions on AWS (S3, Redshift, RDS) and ensure high availability and performance.
- Automate workflow orchestration with Airflow, monitor job health, and troubleshoot failures.
- Collaborate with data scientists and business analysts to understand data requirements and deliver clean, well‑documented datasets.
- Apply data quality checks, lineage tracking, and security best practices across the data ecosystem.
Requirements
- 3+ years of experience in data engineering or related field.
- Hands‑on experience with AWS services (S3, Redshift, Glue, Lambda).
- Knowledge of containerization (Docker) and CI/CD pipelines.
- Strong problem‑solving skills and ability to work in a fast‑paced environment.
Skills
pythonsqlapache sparkawsdockerairflow