onsite
Senior Platform Engineer - Big Data Platform - Conrad Electronic SE
Devops Engineer
Lead the design, deployment, and optimization of a scalable Big Data platform using Hadoop, Spark, and cloud services, ensuring high availability, performance, and security for enterprise analytics workloads.
About the role
Key Responsibilities
- Architect and maintain a fault‑tolerant Big Data platform on AWS, integrating Hadoop, Spark, and related ecosystem components.
- Design and implement data ingestion pipelines, ETL processes, and real‑time streaming solutions using Python and Scala.
- Automate deployment, scaling, and monitoring with Kubernetes, Helm, and CI/CD pipelines.
- Collaborate with data scientists and business analysts to optimize query performance and resource utilization.
- Ensure platform security, compliance, and disaster recovery strategies are in place and continuously improved.
Requirements
- 5+ years of experience in Big Data engineering and platform operations.
- Strong proficiency in Hadoop ecosystem, Spark, and Python/Scala programming.
- Hands‑on experience with AWS services (EMR, S3, EC2, EKS) and container orchestration.
- Solid understanding of data modeling, performance tuning, and distributed systems.
- Excellent problem‑solving skills and a proactive, collaborative mindset.
Skills
apache sparkpythonscalaawskubernetes