onsite
Staff Machine Learning Engineer, Voice AI - Together AI
ML Engineer
Staff Machine Learning Engineer sought for a Voice AI platform focused on optimizing model serving infrastructure. Expertise in inference engines, GPU optimization, and latency reduction is crucial.
About the role
Key Responsibilities
- Drive the model serving layer for voice AI workloads, focusing on production-grade, real-time applications.
- Work hands-on with inference engines such as TRT-LLM and SGLang to optimize model serving.
- Optimize models like Whisper, Parakeet, Orpheus, and Kokoro for best-in-class latency and reliability.
- Profile GPU utilization and design batching strategies to maximize throughput.
- Push the boundaries of inference performance for voice AI applications.
Requirements
- Proven experience as a Machine Learning Engineer, with a focus on model serving.
- Hands-on experience with inference engines like TRT-LLM and SGLang.
- Strong understanding of GPU utilization profiling and optimization techniques.
- Proficiency in Python and experience with cloud platforms like AWS.
- Familiarity with speech-to-text and text-to-speech models is highly desirable.
Skills
machine learningpythonaws