remote
Data Engineer - Australian Securities and Investments Commission
Data Engineer
Join a government data engineering team to design, build, and maintain scalable data pipelines and warehouses, leveraging Python, SQL, cloud services, and orchestration tools to support enterprise‑wide analytics.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL pipelines to ingest, transform, and load data from multiple source systems.
- Build and optimise data warehouse schemas and data models to support reporting and analytics across the organisation.
- Implement and manage workflow orchestration using Apache Airflow or similar tools.
- Collaborate with data analysts, scientists, and business stakeholders to understand data requirements and deliver reliable data solutions.
- Monitor, troubleshoot, and improve performance of data pipelines and infrastructure in a cloud environment (AWS/Azure).
Requirements
- Proven experience with Python and SQL for data manipulation and pipeline development.
- Hands‑on experience building and maintaining data warehouses or data lakes (e.g., Snowflake, Redshift, BigQuery).
- Familiarity with cloud platforms (AWS or Azure) and related services such as S3, Lambda, Data Factory, or equivalent.
- Experience using orchestration tools like Apache Airflow, Prefect, or similar.
- Strong problem‑solving skills and ability to work collaboratively in a cross‑functional team.