onsite
Platform Engineer - Analytics & AI Platform - REWE digital GmbH
Devops Engineer
Design and operate scalable analytics and AI platforms, building data pipelines, real‑time streaming, and machine‑learning services on cloud-native infrastructure.
About the role
Key Responsibilities
- Architect, develop, and maintain a cloud‑native analytics and AI platform supporting batch and real‑time data processing.
- Build and optimize data pipelines using Python, Apache Spark, and Kafka to ingest, transform, and serve large‑scale datasets.
- Implement CI/CD pipelines, container orchestration (Kubernetes), and infrastructure‑as‑code on AWS to ensure reliable, repeatable deployments.
- Collaborate with data scientists and product teams to integrate machine‑learning models and analytics services into production.
- Monitor platform performance, troubleshoot issues, and continuously improve reliability, scalability, and security.
Requirements
- Strong experience with Python and big‑data frameworks such as Apache Spark.
- Hands‑on knowledge of streaming technologies (Kafka) and cloud platforms (AWS).
- Proficiency in containerization (Docker) and orchestration (Kubernetes) with CI/CD tooling.
- Solid understanding of data engineering concepts, ETL processes, and machine‑learning model deployment.
- Ability to work autonomously in an agile environment and communicate technical solutions effectively.
Skills
pythonapache sparkkafkaawskubernetescicd