remote
Testing/Quality Architect for AI - Stellent IT LLC
Software Engineer
Lead enterprise‑wide AI testing transformation, designing and implementing robust test architectures for generative AI models, ensuring quality, compliance, and continuous delivery across the organization.
About the role
Key Responsibilities
- Architect and implement end‑to‑end testing strategies for generative AI models, covering data, inference, and deployment pipelines.
- Design scalable test automation frameworks using Python and modern CI/CD tools to enable rapid, repeatable validation.
- Collaborate with data scientists, ML engineers, and product teams to define quality metrics, test coverage, and risk mitigation plans.
- Lead the adoption of AI‑specific testing practices, including model drift detection, bias assessment, and explainability verification.
- Mentor and coach cross‑functional teams on best practices in AI quality engineering and continuous improvement.
Requirements
- 5+ years of experience in software quality engineering with a focus on AI/ML systems.
- Deep knowledge of generative AI technologies (e.g., GPT, diffusion models) and their testing challenges.
- Proficiency in Python, test automation frameworks, and CI/CD pipelines.
- Strong analytical skills and experience with model validation, bias detection, and performance monitoring.
- Excellent communication and leadership abilities to drive enterprise‑wide quality initiatives.
Skills
generative aitest automationcicdpython