Lead Applied Scientist

About the Role

As an Applied Scientist at Prolific, you will be instrumental in designing and prototyping AI/ML methods that enhance data quality, scale human judgement, and bolster robust AI evaluation workflows. This role is focused on applied problems such as quality modelling, judgement aggregation, evaluation design, LLM-assisted review, and reliability testing for AI systems. It is ideal for individuals with profound scientific judgement, strong applied ML skills, and a pragmatic approach to methods that are effective in real customer and product environments. This position is distinct from pure research or production ML engineering; you will transform ambiguous challenges into clear methodologies, benchmarks, models, and prototypes for adoption by product and engineering teams.

What You'll Be Doing

• Prototype AI/ML methods to improve human data quality, judgement aggregation, and AI evaluation workflows.
• Design experiments, benchmarks, and reliability tests to measure the effectiveness of new methods in enhancing quality, efficiency, or customer outcomes.
• Apply classical ML, statistics, LLMs, and agentic techniques where they offer practical value.
• Utilise modern AI tools to accelerate prototyping, experimentation, and iteration.
• Collaborate closely with product and engineering teams to translate scientific methods into scalable platform capabilities.
• Communicate technical assumptions, trade-offs, and recommendations clearly to both technical and non-technical teams.

What We are looking for

• PhD or MSc in Computer Science, Mathematics, Statistics, Machine Learning, or a related field.
• 3+ years of experience in applied ML, AI research, or data science with a proven track record of real-world impact.
• Experience with human-in-the-loop AI systems, including RLHF, annotation pipelines, data quality modelling, judgement aggregation, benchmarks, or AI evaluation.
• Fluency with modern LLM and agentic techniques, such as Retrieval-Augmented Generation (RAG), LLM-as-judge, multi-agent workflows, synthetic data generation, and automated quality review.
• Strong Python skills and the ability to rapidly build, test, and iterate on working prototypes.
• Sound judgement in selecting appropriate methods, whether simple statistical, classical ML, LLMs, or agentic approaches.
• Ability to translate ambiguous product or customer problems into clear hypotheses, experiments, metrics, and reusable methodologies.
• Strong cross-functional communication and experience partnering with product and engineering teams.