Job Description
We are looking for a highly technical, hands-on Data Infrastructure Engineer to join our team. This is not a standard Data Engineering role; instead, you will focus on building robust, scalable Data Infrastructure platforms using Open Source software. Your primary mission is to deploy and manage high-performance data technologies—such as Apache Pinot, Apache Druid, and Trino—within a well-architected Amazon EKS environment.
You will work on pioneering, high-risk technical solutions, navigating complex customer environments to deliver cutting-edge infrastructure that pushes the boundaries of standard implementations.
Responsibilities
Key Responsibilities
- Infrastructure Implementation: Architect and deploy Open Source Data OSS (Trino, Spark, Pinot, Druid) on Amazon EKS.
- Hands-on Development: Maintain a 100% hands-on approach, utilizing IDEs, Git, and strict coding best practices to manage infrastructure as code.
- Problem Solving: Navigate complex customer hierarchies and technical landscapes to identify, propose, and implement bespoke solutions.
- Pioneering Work: Design and implement novel, high-risk solutions that have not been previously established.
- Collaboration: Act as a subject-matter expert in Open Source technologies, ensuring all builds are well-architected and production-ready.
QUALIFICATIONS REQUIRED
- Agree with our Corporate Values Statement.
- Believes in the company's mission: to help people.
- Believes in the principles of open source.
- Advanced to fluent English communication skills are essential for this role.
- Kubernetes/EKS: Deep expertise in Kubernetes and Amazon EKS is a must-have.
- Core Infrastructure Tooling: Proficiency in Terraform, Helm, Docker, and Karpenter.
- Data OSS Expertise: Minimum rating of 3 (on a 5-point scale) in deploying and managing Data OSS software, including Trino, Spark, or Pinot.
- Data Foundations: Strong familiarity with Data Lakes, Spark, and Hive architectures.
- Versatility: A demonstrated willingness to work with non-AWS and broad Open Source ecosystems.
- Soft Skills: High comfort level engaging with complex customer environments and an aptitude for high-stakes technical troubleshooting.
NICE TO HAVE
- Prior hands-on experience working with Fruitstand.
- Demonstrated history of contributing to or managing complex open-source projects in production environments.
Originally posted on Himalayas