remote
Data Engineer - VicRoads
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and lakehouse solutions using Python, SQL, Spark, and AWS services to support Victoria’s transportation data initiatives.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load large volumes of transportation and licensing data into cloud data platforms.
- Implement data quality checks, monitoring, and alerting to ensure high data integrity and availability.
- Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver actionable insights.
- Optimize performance of ETL processes using Spark, SQL, and AWS Glue, and implement cost‑effective storage solutions.
- Document data models, pipeline architecture, and best practices for future maintenance and scalability.
Requirements
- 3+ years of experience in data engineering with a strong foundation in Python and SQL.
Skills
pythonsqlawsapache spark