onsite
Data Engineer - eClever Systems GmbH
Data Engineer
Data Engineer responsible for designing, developing, and optimizing data infrastructure, pipelines, and storage systems using Python, SQL, Spark, Airflow, and AWS to support data science and business analytics.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and storage solutions to support analytics and data science initiatives.
- Collaborate with stakeholders, including Data Scientists and business units, to translate requirements into robust data architectures.
- Implement and optimize ETL processes using Python, SQL, and Spark, ensuring data quality and performance.
- Automate and monitor data workflows with Airflow, managing scheduling, error handling, and alerting.
- Leverage AWS services (S3, Redshift, Glue, EMR) to build and scale data infrastructure.
- Continuously improve data processes, documentation, and best practices within an agile environment.
Requirements
- Proven experience as a Data Engineer or similar role, with strong hands‑on skills in Python, SQL, and Spark.
- Solid understanding of ETL concepts, data modeling, and data warehousing.
- Experience with workflow orchestration tools such as Apache Airflow.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, EMR).
- Strong analytical mindset, problem‑solving skills, and ability to work collaboratively in an agile team.
Skills
pythonsqlapache sparkaws