This is a remote position.
Reporting to Head of Engineering, leads Accordion's centralized Quality Engineering and AI Validation function across software delivery pods. Owns the quality operating model, release-readiness standards, test strategy, AI validation framework, and quality metrics used to support both conventional software and AI-enabled solutions. Ensures quality is designed into delivery from the start, not inspected in at the end.
Key Responsibilities
Quality Strategy & Governance
- Define the firm-wide quality strategy, release gates, and validation standards across development pods.
- Establish a risk-based quality model for traditional software and AI-enabled workflows.
- Define when pod-level validation is sufficient and when independent QA validation is required.
AI Validation Leadership
- Build and operationalize the QA approach for AI-led products, including benchmark datasets, scoring rubrics, regression comparisons, and grounded-output validation.
- Establish testing expectations for instruction adherence, consistency, business correctness, control compliance, and hallucination risk.
- Partner with Engineering and Product leaders on quality implications of prompt, model, workflow, and tooling changes.
Team Leadership
- Lead and develop quality engineers, AI quality analysts, and domain-oriented QA resources.
- Improve the maturity of automation, evaluation routines, test evidence standards, and release discipline.
- Create a scalable central model that supports pods without becoming a bottleneck.
Finance & Risk Alignment
- Partner with Finance and practice SMEs to ensure solutions are validated against real business use, materiality, and control expectations.
- Ensure high-risk workflows receive the right level of domain review before production release.
Production Quality
- Define the framework for production quality monitoring, including escaped defects, output-quality degradation, reviewer overrides, and control failures.
- Create visibility into quality trends, readiness, and recurring failure patterns for Engineering and leadership.
Requirements
- 8+ years of QA, software testing, or quality engineering experience, including team leadership.
- Experience building test strategy, release-readiness processes, and automation programs in modern software environments.
- Experience supporting AI-enabled applications, workflow automation, data products, or decision-support systems.
- Strong understanding of functional, integration, regression, API, and data validation approaches.
- Ability to translate business-critical finance workflows into quality controls and release criteria.
- St