onsite
Data Engineer - NSW Government
Data Engineer
Design, build, and maintain scalable data pipelines and warehouses to support cancer research analytics, leveraging Python, SQL, and modern ETL tools within a cloud‑enabled environment.
About the role
Key Responsibilities
- Develop, test, and deploy robust ETL pipelines to ingest, transform, and load clinical and research data from multiple sources.
- Design and optimise data models and warehouse schemas to enable efficient querying and reporting for scientific teams.
- Implement data quality checks, monitoring, and automated alerts to ensure data integrity and reliability.
- Collaborate with data scientists, analysts, and domain experts to understand requirements and translate them into technical solutions.
- Maintain documentation, version control, and best‑practice standards for all data engineering artefacts.
Requirements
- Proven experience with Python and SQL for data manipulation and pipeline development.
- Hands‑on experience building ETL workflows using tools such as Apache Airflow, Azure Data Factory, or similar.
- Strong understanding of data modelling concepts and relational/columnar database technologies.
- Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and associated data services.
- Excellent problem‑solving skills and ability to work collaboratively in a multidisciplinary research environment.