onsite
Senior DevOps Engineer - Big Data Focus - BWI GmbH
Devops Engineer
Lead the design and operation of scalable Big Data pipelines, integrating cloud, container, and automation technologies to deliver high‑availability data services.
About the role
Key Responsibilities
- Architect, deploy, and maintain large‑scale Big Data infrastructures on AWS, ensuring performance, security, and cost efficiency.
- Implement and manage containerized services with Docker and Kubernetes, automating deployments via CI/CD pipelines.
- Collaborate with data engineering teams to optimize data ingestion, processing, and storage workflows.
- Monitor system health, troubleshoot incidents, and continuously improve observability and alerting.
- Document best practices, design patterns, and operational procedures for cross‑functional teams.
Requirements
- 5+ years of DevOps experience with a strong focus on Big Data environments.
- Proficient in AWS services (EMR, S3, Redshift, Glue) and container orchestration (Kubernetes).
- Hands‑on scripting in Python and experience with CI/CD tools (Jenkins, GitLab CI, ArgoCD).
- Solid understanding of data pipeline concepts, ETL processes, and data lake architectures.
- Excellent problem‑solving skills and a proactive, collaborative mindset.
Skills
awskubernetesdockercicdpython