remote
Data Engineer - AI Spark, Databricks and Healthcare - Cotiviti
Data Engineer
Data Engineer – AI focused on Spark and Databricks, driving advanced data pipelines, validation, and optimization for healthcare analytics.
About the role
Key Responsibilities
- Develop, maintain, and execute intermediate to advanced Spark scripts for data management, validation, and integration across healthcare datasets.
- Write and optimize basic to intermediate SQL queries to support data quality checks and reporting.
- Improve query performance and overall pipeline efficiency through profiling, tuning, and resource management.
- Analyze data flows to identify anomalies, data quality issues, and opportunities for process improvement.
- Collaborate with cross‑functional teams to design scalable data solutions and ensure compliance with security and privacy standards.
Requirements
- Proven experience with Apache Spark and Databricks in a production environment.
- Strong SQL skills and familiarity with relational database design.
- Hands‑on knowledge of ETL concepts, data modeling, and data quality practices.
- Experience working with healthcare or regulated data is a plus.
- Excellent problem‑solving skills and ability to communicate complex technical concepts clearly.