onsite
Senior Data Engineer - Google Cloud Platform - Schwarz Gruppe
Data Engineer
Lead the design and implementation of scalable data pipelines on Google Cloud, leveraging Kafka, Spark, and Python while ensuring robust container orchestration with Kubernetes and Docker.
About the role
Key Responsibilities
- Architect, develop, and maintain high‑throughput data pipelines on Google Cloud Platform, ensuring reliability and performance for real‑time analytics.
- Implement streaming solutions using Apache Kafka, integrating with batch and real‑time processing frameworks such as Spark.
- Design and manage containerized services with Docker and Kubernetes, automating deployment, scaling, and monitoring.
- Collaborate with data scientists and product teams to translate business requirements into efficient data models and ETL processes.
- Optimize data workflows for cost, latency, and resource utilization across the cloud environment.
Requirements
- Extensive experience in data engineering, big data, or software engineering, preferably in a senior role.
- Deep expertise in Google Cloud Platform services (BigQuery, Dataflow, Pub/Sub, Cloud Storage).
- Proficiency in Python and Spark for data processing and transformation.
- Hands‑on experience with Apache Kafka, Kubernetes, and Docker for streaming and orchestration.
- Strong analytical skills, problem‑solving mindset, and excellent communication abilities.
Skills
pythonkubernetesdocker