Site Reliability Engineer
We run nearly 30 models across heterogeneous hardware, and keeping that fleet fast, reliable, and cost-effective is a serious engineering challenge.
About the Role
We're looking for a Staff Site Reliability Engineer who is equally at home writing production software and running the infrastructure it lives on — and who wants to take ownership of one of the hardest, highest-leverage problems on our platform: intelligently managing a large fleet of GPU-backed models.
We run nearly 30 models across heterogeneous hardware, and keeping that fleet fast, reliable, and cost-effective is a serious engineering challenge. You'll build the GPU management and scheduling platform that sits at the center of it — collecting utilization and load metrics, interpreting what they actually mean, and using them to make real-time decisions about admission control and scaling. The goal: route and schedule inference calls so we use our capacity efficiently without exceeding it, and scale model replicas up and down automatically as demand shifts.
This is a senior role for someone with a decade in the field who can move fluidly between systems engineering and software development, and who is excited to own a complex, evolving system end to end.
What You'll Do
Design and build our GPU management and scheduling platform — the system that decides when, where, and how inference calls run across a fleet of ~30 models on heterogeneous hardware
Build the metrics pipeline that collects GPU load and utilization data, and the logic that turns those signals into decisions
Implement admission control to protect capacity — deciding when to accept, queue, or shed inference requests so we operate within fleet limits
Build autoscaling that adjusts the number of model replicas in response to real-time demand and utilization
Develop cloud orchestration systems and operators in Python and Go to manage the model fleet
Architect and operate scalable, fault-tolerant, secure production systems on AWS, GCP, or Azure
Design and build infrastructure automation and deployment pipelines (Terraform, CI/CD) as first-class software
Stand up and maintain monitoring, logging, and alerting that keep the platform reliable and performant
Develop and enforce security and compliance policies appropriate to a healthcare AI platform
Partner with engineers and research scientists to diagnose and resolve complex infrastructure, deployment, and operational issues
Mentor engineers and raise the technical bar across the team
We value in-person teamwork and believe the best ideas happen together. Our team is expected to be in the office five days a week in Gurugram. This role will also require engaging with a US based team requiring some flexibility for overlapping hours with California.
What You Bring
Must-Have
10+ years of professional experience across site reliability / DevOps engineering and software engineering
Computer Science Degree Required from a top CS program.
Posted November 4, 2026