QA Engineer, AI - TrueLearn

Job Title: QA Engineer, AI

FLSA status: Regular Full-Time Remote (US-Based)

Reports to: QA Manager

About TrueLearn

TrueLearn is an educational software company headquartered in Charlotte, NC, dedicated to empowering medical institutions and individual learners to maximize performance on high-stakes licensure exams. Through a suite of innovative, learning science–driven tools, TrueLearn supports clinicians at every stage of their careers — from board preparation through continuing education.

To date, TrueLearn has prepared over 70,000 healthcare professionals in their pursuit of board certification and works closely with hundreds of medical and health science institutions and residency programs throughout the United States. Our learning science–driven platform supports clinicians at every stage of their careers, from board preparation through continuing education. The company is rapidly growing, continuously innovating its technology platform and expanding its portfolio to meet the evolving needs of the healthcare community.

At the heart of everything TrueLearn does is a simple but powerful truth: the work touches the people who touch patients, which means quality here is never cosmetic. Every line of code, every model, and every experiment has the potential to ripple outward into better care, better outcomes, and better lives.

TrueLearn's culture is built around authenticity, empowerment, and a relentless commitment to advocating for learners. Strong core values guide decisions, priorities, and tradeoffs as the company scales - ensuring that growth never comes at the cost of alignment, integrity, or impact on healthcare professionals and patient care.

We are looking for a QA Engineer to help us protect and raise the quality of our learning experiences. This is a hands-on role where you will work day-to-day inside an engineering team - close to the code, the product, and the learners we support.

Position Summary

Testing AI features presents new challenges, but at its core, it relies on solid testing judgment. Your job is to help us determine what "good" looks like and to catch issues before a learner sees them. Your work spans three areas:

• Testing AI-powered features. You work out how we measure whether an AI feature is actually good — building test sets, defining what “good” means when there’s no single right answer, and catching it when quality slips as prompts or models change. You learn to tell normal AI variation apart from a real bug.
• Using AI to test better. You use AI tools in your own work — to help write test cases, build automation, sort through failures, and generate test data — and you build a clear sense of where they help and where they mislead.