onsite
LLM Performance Operation - Specialist
LLM Performance Operation - Specialist
As an LLM Performance Operation Specialist, you will be a core member of the LLM Global Data Team, focusing on measuring and analyzing the performance of LLMs. This involves creating and conducting tests, supporting auto-evaluation development, and collaborating with product managers and external experts to enhance performance analysis.
About the role
About the Role
As a core member of our LLM Global Data Team, you'll be at the heart of our operations. Gain first-hand experience in understanding how to measure and analyze performance of LLM on various dimensions.
Your Role Will Involve
- Gaining direct experience in LLM and LLM-relevant product performance measurement, including creating and conducting tests, to help tech and product teams to understand LLM and product performance.
- Supporting the development of auto-evaluation and testing.
- Collaborating with product managers and analysts to work with external experts and partners to strengthen performance analysis efforts at scale or in expertise domains.
Qualifications
- Professional proficiency in English with excellent or native-level writing and critique skills. Multilingual candidates are preferred.
- Experience as NLP model and search product performance evaluators is strongly preferred.
- Candidates should be intelligent, diligent, and skilled in comparing and analyzing content following a framework.
- Comfort with meeting deadlines and paying attention to details.
- A deep interest in LLMs.