This is a remote position.
Reporting to Manager, Quality Engineering & AI Validation, designs and scales Accordion's automated quality framework across products, services, and AI-enabled workflows. Owns the architecture behind automated validation, test harnesses, evaluation pipelines, test data strategy, and cross-pod tooling standards.
- Design and implement the core automation framework for functional, API, integration, regression, and workflow testing.
- Define standards for reusable test assets, environment usage, data setup, and evidence capture.
- Create scalable approaches to testing complex, multi-system workflows.
AI Evaluation Infrastructure
- Build and maintain regression harnesses for AI-enabled features, including prompt and workflow comparisons, scored evaluation runs, and benchmark execution.
- Partner with AI quality analysts to support rubric-based testing and measurable release criteria.
Test Data & Environment Strategy
- Define approaches for test data, synthetic data, masked data, and scenario generation.
- Improve reliability and repeatability of validation environments.
Engineering Partnership
- Work closely with software engineers to improve testability, observability, CI/CD integration, and defect prevention.
- Help pods move quality checks earlier in the development lifecycle.
Requirements
- 7+ years of experience in QA automation, SDET, or quality engineering roles.
- Strong experience with test automation frameworks, API testing, CI/CD, and integration validation.
·Strong scripting or coding skills.
- Experience working with data-intensive or workflow-heavy applications.
- Exposure to AI or LLM validation tooling or strong interest in building in that space.
- Ability to design frameworks, not just individual tests.
·Bachelor's degree preferred.
Key Traits for Success
·Systems-minded and deeply technical.
- Focused on scale, repeatability, and leverage.
- Comfortable setting standards others adopt.
- Practical about balancing sophistication with delivery pace.
Benefits
Actual compensation packages are determined by evaluating a wide array of factors unique to each candidate, including but not limited to skill set, years and depth of experience, education, certifications, cost of labor, and internal equity.
Originally posted on Himalayas