onsite
Senior Data Engineer Scala/Java, Spark, AWS - Commonwealth Bank of Australia
Data Engineer
Senior Data Engineer driving a Data Transformation program, leveraging Scala, Java, Spark, and AWS to architect scalable data pipelines, model data, and elevate data quality and availability for a next‑generation banking platform.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines using Scala, Java, and Spark on AWS.
- Architect data models and schemas to support analytics, reporting, and machine learning workloads.
- Collaborate with cross‑functional teams to define data quality standards and implement best practices.
- Optimize performance and cost of data processing workflows in a cloud environment.
- Mentor junior engineers and promote a culture of continuous improvement and innovation.
Requirements
- 5+ years of experience in data engineering with strong proficiency in Scala and Java.
- Hands‑on expertise with Apache Spark, including batch and streaming processing.
- Deep knowledge of AWS services (EMR, S3, Glue, Redshift, Lambda, etc.) and cloud architecture patterns.
- Solid understanding of data modeling, ETL design, and data quality principles.
- Excellent problem‑solving skills and a passion for building scalable, reliable data solutions.