remote
Senior Data Engineer - AI Platforms - Optum
Data Engineer
Senior Data Engineer focused on building and scaling AI‑ready data platforms, leveraging Python, Spark, SQL, and cloud services to enable advanced analytics and machine learning pipelines.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines that ingest, transform, and store large‑volume health data for AI and analytics use cases.
- Implement robust data models and schemas in cloud data warehouses (e.g., Snowflake, Redshift) to support downstream machine‑learning workloads.
- Orchestrate workflows using Apache Airflow and ensure reliable data delivery with streaming technologies such as Kafka.
- Collaborate with data scientists, product owners, and engineering teams to translate business requirements into performant data solutions.
- Apply best practices for data security, governance, and compliance within a regulated healthcare environment.
Requirements
- 5+ years of hands‑on experience as a Data Engineer, preferably in a healthcare or AI‑focused setting.
- Proficiency in Python, SQL, and big‑data processing frameworks like Apache Spark.
- Strong experience with AWS services (S3, EMR, Glue, Lambda, etc.) and cloud‑native data warehousing.
- Hands‑on knowledge of workflow orchestration (Airflow) and streaming platforms (Kafka).
- Demonstrated ability to design scalable, fault‑tolerant data architectures and to work collaboratively in cross‑functional teams.
Skills
pythonsqlapache sparkawskafka