remote
Staff Data Engineer - Commonwealth Bank of Australia
Data Engineer
Lead the design and delivery of scalable data pipelines and regulatory reporting solutions using Python, Spark, SQL, and AWS, driving compliance and innovation across the bank’s global data platform.
About the role
Key Responsibilities
- Architect, build, and maintain large‑scale data pipelines that ingest, transform, and deliver regulatory reporting data across the enterprise.
- Collaborate with data scientists, product owners, and compliance teams to define data requirements and ensure data quality and integrity.
- Leverage AWS services (Glue, Redshift, S3, Lake Formation) and Spark to optimize performance, cost, and scalability of data workflows.
- Implement robust data modeling, metadata management, and governance practices to support regulatory compliance and audit readiness.
- Mentor and lead a high‑performing data engineering team, fostering a culture of continuous improvement and technical excellence.
Requirements
- 10+ years of experience in data engineering, with a proven track record in building enterprise‑grade data platforms.
- Deep expertise in Python, Apache Spark, SQL, and AWS data services.
- Strong understanding of regulatory reporting frameworks and data governance principles.
- Excellent problem‑solving skills, with the ability to translate complex business requirements into scalable technical solutions.
- Experience leading and mentoring technical teams in a fast‑paced, collaborative environment.
Skills
pythonapache sparksqlaws