Senior Applied Scientist, Multilingual AI Evaluation - Apple

AI systems are only as trustworthy as the methods used to evaluate them. At Apple, where AI powers experiences for billions of people around the world, getting evaluation right is not a support function-it is a foundational science. Our team, part of Apple Services Engineering, is building that scientific foundation: rigorous, scalable evaluation methodology for LLMs, agentic systems, and human-AI interaction. We’re looking for a senior applied scientist to make the evaluation tooling we build work across every language and culture Apple serves. This is a role for someone who is fluent in both modern AI and the science of language, and who can set direction and drive initiatives independently, not just execute them. You’ll do this on a deeply interdisciplinary team working alongside ML researchers, measurement scientists, and platform engineers. Description In this role, you’ll help ensure Apple’s AI features work well across languages and cultures. Your goal is to make our evaluation tooling multilingual from the start so that engineers building AI features can design, test, and ship across the world from day one. It’s a broad applied science role: you’ll shape how Apple evaluates AI wherever the hardest questions are, and you’ll have the opportunity to publish novel work. The scientific challenge is real. How do we ensure we consistently evaluate AI features across different grammar, script, or cultural norms and how do we do this at scale? You’ll bring linguistic judgment to questions like these and, working with measurement scientists and ML researchers, turn it into validated methodology that holds across dozens of languages. This is a hands-on role. You’ll design and implement your own methods in Python, working closely with research and engineering partners, while staying focused on the science of getting evaluation right. ","responsibilities":" Extend Apple’s AI evaluation methodology and tooling to new languages and locales, so our AI experiences are equally capable, accurate, and culturally appropriate - not just translated English. Design and validate evaluation methods, benchmarks, and metrics that capture language- and culture-specific phenomena: grammar, morphology, script, register, dialect, code-switching, and cultural norms. Build and curate high-quality multilingual datasets and human evaluation protocols, partnering with linguists and native-speaker annotators. Investigate how LLMs and agentic systems behave across languages, identifying systematic failure modes, capability gaps, and quality disparities between high- and low-resource languages. Partner with engineers to productionize your methods so they run reliably and at scale, implementing your own work in Python. Communicate findings clearly - translating results into actionable guidance for model and product teams, and publishing novel work where appropriate. Preferred Qualifications PhD in Linguistics, Computational Linguistics, or NLP with a focus on multilingual or cross-lingual modeling. Publications in NLP, multilingual evaluation, or evaluation methodology. Hands-on experience with modern ML frameworks (PyTorch, JAX) and with fine-tuning or evaluating LLMs. Experience with low-resource languages, dialectal variation, or sociolinguistics. Familiarity with localization/internationalization workflows and quality assessment. Experience with LLM-as-judge approaches, rubric design, or bias and fairness evaluation across languages. Fluency or professional proficiency in one or more languages in addition to English. Minimum Qualifications MS in Linguistics, Computational Linguistics, NLP, Computer Science, or a related field - or equivalent research/work experience. Deep expertise in linguistics, with working fluency in the structure of multiple languages beyond English. Strong proficiency in Python. Solid understanding of LLMs and AI evaluation fundamentals, including how language models process and generate across languages. Demonstrated experience shipping or evaluating features across multiple languages or locales. Experience designing benchmarks, datasets, or human evaluation protocols, with attention to statistical rigor and reproducibility. Ability to drive initiatives independently and collaborate across a cross-functional, interdisciplinary team. Strong written and verbal communication skills. Pay & Benefits At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $205,400 and $308,500, and your base pay will depend on your skills, qualifications, experience, and location. Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.