hybrid
Engineering Manager, Evaluation Metrics and Sampling
Engineering Manager, Evaluation Metrics and Sampling
Waymo is seeking an experienced, technically hands-on Engineering Manager to lead a small team focused on scaling evaluation and ML development. This role involves developing tools for data mining and sampling, building automation for scaling to new cities, and providing technical leadership in both engineering and data science projects.
About the role
About the Role
The Waymo Driver is performing well in more and more cities, with our scale rapidly accelerating. Our team’s job is to ensure that we can continue to evaluate safety and service quality across new cities and countries, each with their unique challenges. We need an experienced, technically hands-on engineering manager with strong quantitative skills to lead a small team and work with partners to scale eval and ML development by exploring new methods and delivering tools for mining and sampling data. In this hybrid role, you will report to a Senior Staff manager.
Responsibilities
- Develop tools for easy mining of novel situations that might impact safety from our driving logs for use in evaluation and ML datasets, using a mixture of ML and structured data
- Advanced sampling techniques and tuning of simulations to get strong signal for metrics and models from our datasets
- Build automation for our mining and sampling ecosystem so Waymo can rapidly scale to new cities
- Make hands-on technical contributions to both engineering and data science-oriented projects
- Provide strong management and technical leadership for a small team of data science-friendly engineers
- Partner with leads and teams across Waymo to determine and execute on strategy, including product, engineering, data science, frontend, and backend
Requirements
- Strong engineering design and programming skills
- Strong data fluency and analysis skills
- Familiarity with statistical methods including hypothesis testing, experiment design, and sampling
- Familiarity with foundational and contemporary ML concepts and practices
- Experience managing teams of up to 8 people
- BS/MS in Computer Science, Robotics, Statistics, Physics, Math, or other quantitative area
- 7+ years of industry experience with:
- C++, SQL, and Python experience
- Navigating and modifying a large code base containing C++, SQL, and Python
- Building data processing pipelines
Preferred Qualifications
- Eval experience, such as building and validating metrics to measure quality in complex systems
- Strong statistical background (MSc-level stats or similar expertise from work experience)
- Strong ML background (MSc-level ML or similar expertise from work experience)
- Experience working in the AV industry and/or with simulations