onsite
Senior Data Engineer - Westpac Group
Data Engineer
Senior Data Engineer designing, building, and optimizing scalable data pipelines and solutions using Python, SQL, Spark, and AWS to power enterprise-wide transformation and deliver reliable data products.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load data across multiple sources into a unified data lake.
- Collaborate with cross‑functional teams to translate business requirements into scalable data solutions, ensuring performance, reliability, and security.
- Implement best practices for data modeling, metadata management, and data quality to support analytics and reporting.
- Optimize existing workflows using Spark, SQL, and AWS services (Glue, Redshift, S3) for cost‑effective, high‑throughput processing.
- Mentor junior engineers and contribute to the evolution of the data architecture strategy.
Requirements
- 5+ years of experience in data engineering with a strong background in Python, SQL, and Spark.
- Hands‑on experience with AWS data services (Glue, Redshift, S3, Athena).
- Proven ability to design and implement data models, ETL processes, and data lake architectures.
- Strong analytical skills and a passion for building scalable, maintainable data solutions.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlapache sparkaws