onsite
Data Engineering & Modelling Lead - Transport for NSW
Software Engineer
Lead the design and delivery of high‑performance data engineering and modelling solutions, driving scalable data pipelines and advanced analytics for transport services using SQL, Python, Spark and AWS.
About the role
Key Responsibilities
- Architect, develop and maintain end‑to‑end data pipelines that ingest, transform and store large volumes of transport data.
- Design and implement robust data models and schemas to support analytics, reporting and decision‑making.
- Collaborate with cross‑functional teams to define data requirements, quality standards and performance metrics.
- Leverage AWS services (Glue, Redshift, S3) and Spark to build scalable, fault‑tolerant data workflows.
- Mentor junior engineers, promote best practices in coding, testing and documentation.
Requirements
- Proven experience in data engineering, with strong SQL and Python skills.
- Hands‑on expertise with Apache Spark and AWS data services.
- Solid understanding of data modelling, ETL processes and data governance.
- Excellent problem‑solving abilities and a collaborative mindset.
- Experience in a public sector or large‑scale transport environment is a plus.
Skills
sqlpythonapache sparkaws