remote
Software Development Engineer, Open Data Analytics Fundamentals - Amazon Web Services
Software Engineer
Software Development Engineer responsible for building, validating, and distributing AMIs and Docker images that power Amazon EMR, AWS Glue, and Athena across EC2, EKS, and Serverless deployment modes.
About the role
Key Responsibilities
- Design, develop, and maintain the automated pipelines that build, test, and publish AMIs and Docker images for EMR, Glue, and Athena.
- Collaborate with cross‑functional teams to integrate new open‑source data processing frameworks (e.g., Spark, Hive, Presto) into the release lifecycle.
- Implement scalable, fault‑tolerant solutions for multi‑mode deployments (EC2, EKS, Serverless) using AWS services.
- Drive continuous improvement of CI/CD processes, monitoring, and performance metrics for the Kamino pipeline.
- Participate in code reviews, troubleshooting, and root‑cause analysis to ensure high reliability of production clusters.
Requirements
- Strong programming experience in Java and Python.
- Deep understanding of AWS infrastructure, including EC2, EKS, and serverless services.
- Hands‑on experience with containerization (Docker) and orchestration (Kubernetes/EKS).
- Proven ability to build and maintain large‑scale CI/CD pipelines for image creation and distribution.
- Solid Linux/Unix skills and familiarity with distributed data processing frameworks.
Skills
javapythonawsdockerkubernetes