onsite
Senior Quality Engineer, Data & AI Platform - M3, LLC
Software Engineer
Senior Quality Engineer focused on data and AI platforms, driving automated testing, data validation, and ML model verification using Python, SQL, Spark, Airflow, and CI/CD pipelines.
About the role
Key Responsibilities
- Design and implement automated test frameworks for data pipelines, ETL processes, and machine‑learning models.
- Develop data validation suites using Python, SQL, and Apache Spark to ensure accuracy, completeness, and performance of large‑scale datasets.
- Collaborate with data engineers, data scientists, and product owners to define quality criteria and acceptance tests for AI‑driven features.
- Integrate testing workflows into CI/CD pipelines (Jenkins, GitLab CI) and orchestrate scheduled runs with Airflow.
- Monitor production data quality, investigate anomalies, and drive root‑cause analysis and remediation.
Requirements
- 5+ years of experience in software quality engineering with a strong focus on data engineering or machine‑learning platforms.
- Proficiency in Python programming, SQL querying, and big‑data tools such as Apache Spark.
- Hands‑on experience with workflow orchestration (Airflow) and CI/CD automation.
- Solid understanding of data validation, data profiling, and ML model testing methodologies.
- Excellent problem‑solving skills and ability to work cross‑functionally in an agile environment.
Skills
pythonsqlapache sparkairflowcicd