The AI Product Manager - Ops will drive operational excellence in data testing, validation, and monitoring of AI models in production. This role involves ensuring reliable, safe, and responsible AI-driven features at scale by managing data quality, production monitoring, testing strategies, and incident resolution, while also collaborating cross-functionally and ensuring compliance.
About the role
Role Description
We are seeking a highly detail-oriented AI Product Manager - Ops to own and drive operational excellence in data testing, validation, and monitoring of AI models in production environments. This role sits at the intersection of product, data science, engineering, and operations. The successful candidate will be responsible for ensuring that AI-driven features perform reliably, safely, and responsibly at scale, while continuously improving product quality and customer experience.
Key Responsibilities
Data Quality & Validation: Design and manage frameworks for data testing in production, ensuring high-quality, unbiased, and consistent datasets feed into AI models.
Production Monitoring: Establish and monitor key health metrics (e.g., drift detection, anomaly detection, precision/recall, latency, and fairness) for AI systems in live environments.
Testing Strategy: Develop and operationalize A/B testing and canary release strategies for model updates to mitigate risk.
Incident Management: Lead root cause analysis and resolution of data-related production issues in collaboration with engineering, ML, and data science teams.
Tooling & Automation: Drive requirements for automated testing pipelines, monitoring dashboards, and alerting systems.
Cross-functional Collaboration: Work closely with product managers, ML engineers, data scientists, SREs, and compliance teams to ensure responsible AI deployment.
Governance & Compliance: Ensure adherence to AI risk management, regulatory guidelines, and internal governance frameworks.
Feedback Loop: Build mechanisms to capture customer and system feedback for continuous model improvement.