onsite
Cloud Data Engineer - Home Shopping Europe
Data Engineer
Lead the design, implementation, and maintenance of scalable cloud data pipelines using Python, AWS, and Spark to support real‑time analytics for a live commerce platform.
About the role
Key Responsibilities
- Design, develop, and optimize data pipelines on AWS (Glue, Redshift, S3) to ingest, transform, and store large volumes of structured and unstructured data.
- Implement Spark jobs in Python for batch and streaming ETL processes, ensuring high performance and fault tolerance.
- Collaborate with data scientists and product teams to expose clean, reliable datasets for analytics, reporting, and machine‑learning models.
- Monitor pipeline health, troubleshoot issues, and continuously improve data quality and processing efficiency.
- Document architecture, data flows, and best practices for the engineering team.
Requirements
- 3+ years of experience in cloud data engineering, preferably on AWS.
- Strong proficiency in Python and Spark for data processing.
- Hands‑on experience with SQL and data warehousing solutions (Redshift, Snowflake).
- Knowledge of CI/CD pipelines, version control (Git), and containerization (Docker).
- Excellent problem‑solving skills and a collaborative mindset.
Skills
pythonawsapache sparksql