Engineering Manager SDET - AI

About HighLevel

HighLevel is an AI-powered, all-in-one white-label sales & marketing platform that empowers agencies, entrepreneurs, and businesses to elevate their digital presence and drive growth. We are proud to support a global and growing community of over 1 million businesses, comprised of agencies, consultants, and businesses of all sizes and industries. HighLevel empowers users with all the tools needed to capture, nurture, and close new leads into repeat customers. As of mid 2025, HighLevel processes over 4 billion API hits and handles more than 2.5 billion message events every day. Our platform manages over 470 terabytes of data distributed across five databases, operates with a network of over 250 microservices, and supports over 1 million hostnames.

Our People

With over 1,500 team members across 15+ countries, we operate in a global, remote-first environment. We are building more than software; we are building a global community rooted in creativity, collaboration, and impact. We take pride in cultivating a culture where innovation thrives, ideas are celebrated, and people come first, no matter where they call home.

Our Impact

As of mid 2025, our platform powers over 1.5 billion messages, helps generate over 200 million leads, and facilitates over 20 million conversations for the more than 1 million businesses we serve each month. Behind those numbers are real people growing their companies, connecting with customers, and making their mark - and we get to help make that happen.

About the Role

We are looking for a highly skilled SDET Manager to spearhead the quality of AI application development at HighLevel. This role is very critical in ensuring the relevancy, consistency, reliability, contextual recall and fairness of our AI platform.

Ideal candidates will have a combination of technical depth in AI/ML, Agentic AI along with deep expertise in test automation covering model validation, agentic workflows, and quality leadership.

Requirements

• Overall 8-12 years of experience in which minimum of 1 year in testing AI products
• Good experience in Prompt Engineering, testing AI applications like conversations AI, voice AI, workflow AI
• Architect and scale test automation frameworks for APIs, micro-services, ML pipelines, and agentic AI workflows
• Experience in testing RAG based applications using frameworks like TruLens, RAGAs
• Leverage AI to improve coverage and efficiency across testing lifecycle
• Hands-on experience testing agentic AI frameworks (e.g., Agentic Studio, LangChain, AutoGen, crewAI)
• Strong programming skills in Python and Java or TypeScript/JavaScript
• Proven expertise in test automation frameworks (Selenium, Playwright, Cypress etc.)
• Experience with cloud platforms (AWS, GCP) and CI/CD pipelines

Responsibilities

• Review testing strategies for AI application and fine tune strategy to minimize issues in production
• Establish evaluation pipelines for detecting hallucination, context retention, latency in multi model environments
• Generate right data set to simulate customer usage scenarios
• Test for data, model and code issues
• Define relevant KPIs for AI application testing
• Non Functional testing - Performance, Scalability, Reliability, Security
• Implement A/B testing at appropriate levels to identify customer impact
• Run down defect triaging for all customer issues to identify root cause and expand test coverage to mitigate gaps
• Ensure quality deliverables from AI teams
• Hire top talent and mentor SDET team on AI testing skills