Role Summary
Project A is ALX’s AI learning platform: LLM products built on hypotheses about how people learn and how AI can help. Some of those hypotheses are wrong; this role’s job is to find out which ones, fast. It centres on uncertainty reduction rather than product vision. You own the Experimentation-Harness as a function — instrument the AI products well enough to evaluate them, source learners for betas (the binding constraint on the whole programme), run the ongoing beta tests, and close the loop: data reviewed, improvements shipped, next PoC out the door. UX validation is captured alongside learning validation, so we know not just whether it teaches but whether people can use it.
Specific Responsibilities
Learner Supply & Instrumentation
- Build a repeatable pipeline of beta learners, engaging the right internal teams to keep them flowing — the constraint that gates everything else, and it rewards hustle over process.
- Instrument the AI products in partnership with the LLMOps Engineer — you instrument; they build the evals over what is captured.
The Experiment Loop
- Own the experiment loop — a backlog of the team’s biggest uncertainties, experiments designed against them, cycle time measured and shrinking.
- Close the loop: data reviewed, UX and learning validation captured, improvements shipped, and the next PoC out the door.
Skill Requirements - Essential
- Product experimentation: you have run it end-to-end, hypothesis, instrumentation, decision. A/B testing or structured product testing on a live product.
- Technical fluency: you can talk concretely about instrumenting a product, read AI-eval results, and hold your own with engineers. You don’t need to code daily.
- Operator ability: recruiting and coordinating real users, running a beta programme, and wrangling stakeholders.
- Desirable (not required): hands-on eval or analytics skills; EdTech experience.
Essential Traits for Success
- You have a scientific mindset. You know what an experiment can and can’t conclude, and your favourite experiments include ones that killed an idea you loved.
- You’d enjoy the question “how would you get 50 beta learners in two weeks with no budget?” enough to start answering it in the interview.
Person Specification/Attributes
- Courage: Willingness to speak up, challenge the status quo, and embrace new challenges.
- Humility: Openness to learning, seeking help when needed, and a focus on serving others.
- Adventure: A passion for setting ambitious goals, tackling difficult tasks, and finding joy in the journey.
- Initiative: Proactive problem-solving, a sense of ownership, and a willingness to go above and beyond.