onsite
Senior Staff ML Platform Engineer
ML Platform Engineer
The Senior Staff ML Platform Engineer will be responsible for designing and operating GPU infrastructure for model hosting and scaling model serving systems. This role involves end-to-end ownership of the model lifecycle, including optimization, deployment, and monitoring across various modalities.
About the role
About the Role
We are seeking a Senior Staff ML Platform Engineer to design and operate our GPU infrastructure for model hosting. This role involves end-to-end ownership of the model lifecycle, from provisioning to scaling, and optimizing inference systems for various modalities.
Responsibilities
- Design and operate GPU infrastructure for model hosting, including provisioning, scheduling, and cost optimization across cloud and on-premise environments.
- Build and scale model serving systems using vLLM, TensorRT-LLM, Triton, or equivalent, supporting real-time inference with strong latency and availability guarantees.
- Implement multi-model routing to serve multiple models across modalities (text, voice, code, vision) on shared infrastructure.
- Own the model lifecycle end to end: download, deploy, serve, monitor, swap, and scale.
- Drive inference optimization including quantization strategies (AWQ, GPTQ), batching, caching, and cold start reduction.
- Build self-service infrastructure platforms where teams provision compute, storage, and model endpoints through APIs and control planes.
- Implement infrastructure-as-code at scale using Terraform, Pulumi, or CDK.
- Build observability and reliability for inference systems: SLIs/SLOs, GPU utilization monitoring, latency tracking, automated capacity planning, and alerting.
- Define platform standards and governance including multi-tenant isolation, cost attribution, and resource quotas.
- Lead architectural design and influence engineering direction across the AI infrastructure stack.
Skills
GPUVllmTensorRT LLMTritonquantizationAWQGPTQTerraformPulumiCDK