onsite
Data Engineer - ACP IT Solutions GmbH
Data Engineer
Build and maintain scalable data pipelines, optimize data workflows, and collaborate with analytics teams using Python, SQL, AWS, Spark, and containerization.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines to ingest, transform, and store large volumes of structured and unstructured data.
- Implement data quality checks, monitoring, and alerting to ensure reliability and performance of data services.
- Collaborate with data scientists and business analysts to understand data requirements and deliver actionable insights.
- Optimize existing data workflows for scalability, cost-efficiency, and maintainability using cloud services and modern tooling.
- Document data models, pipeline architecture, and best practices for internal teams.
Requirements
- Proven experience as a Data Engineer or similar role, with strong proficiency in Python and SQL.
- Hands‑on experience with AWS services (S3, Redshift, Glue, EMR) and big‑data frameworks such as Apache Spark.
- Familiarity with containerization (Docker) and CI/CD pipelines for data workloads.
- Strong analytical skills and ability to troubleshoot complex data issues.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlawsapache sparkdocker