onsite
Staff Software Engineer, Applied AI, Model Quality - Google
Software Engineer
We're looking for a Software Engineer focused on designing and building scalable technical solutions. This lead role requires 8+ years of relevant experience.
About the role
Minimum qualifications:
- Bachelor’s degree or equivalent practical experience.
- 8 years of experience in software development.
- 5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field.
- 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
- 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
- Experience integrating generative AI tools or Large Language Model (LLM) interfaces into workflows.
Preferred qualifications:
- Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- 8 years of experience with data structures and algorithms.
- 3 years of experience in a technical leadership role leading project teams and setting technical direction.
- 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
Responsibilities
- Build both human-powered and Large Language Model (LLM)-powered automated evaluation systems to assess model performance.
- Establish clear metrics to measure aspects like grounding, coherence, safety, and helpfulness.
- Utilize platforms and tools to efficiently run evaluations across different models and datasets.
- Provide actionable insights from evaluations to improve model quality, often in collaboration with research, and cross-functional teams.
- Create tools and systems that make the evaluation process more efficient and effective.
Skills
natural language processing