onsite
IT Consultant IV, Solutions - Data Engineer - Kaiser Permanente
Data Engineer
Senior data engineering consultant driving large‑scale technical initiatives, optimizing performance, and mentoring teams across data, platform, and architecture domains using Python, Spark, SQL, and AWS.
About the role
Key Responsibilities
- Own and deliver complex data engineering projects, ensuring high performance, reliability, and scalability.
- Collaborate with Data Engineering, Platform Engineering, Data Science, and Architecture teams to evaluate solution designs and influence engineering decisions.
- Mentor developers on modern data engineering practices, including best‑practice coding, testing, and deployment.
- Design, implement, and maintain robust ETL pipelines, data models, and data quality frameworks.
- Leverage AWS services (S3, Redshift, Glue, EMR) and Apache Spark to process and transform large datasets.
Requirements
- 10+ years of experience in data engineering or related technical roles.
- Expertise in Python, SQL, and Apache Spark for large‑scale data processing.
- Strong knowledge of AWS data services and cloud architecture.
- Proven ability to mentor and lead technical teams.
- Excellent problem‑solving skills and a track record of delivering high‑quality solutions at scale.
Skills
pythonsqlapache sparkaws