onsite
Data Engineer - NSW Health
Data Engineer
Data Engineer needed to design, build, and maintain scalable data pipelines and warehouses for cancer research, leveraging Python, SQL, Azure Data Factory, and Snowflake to enable advanced analytics and reporting.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL pipelines to ingest, transform, and load clinical and research data from multiple sources.
- Build and optimise data models and warehouse structures in Snowflake to support analytics, reporting, and machine‑learning workloads.
- Implement data integration workflows using Azure Data Factory, ensuring reliability, scalability, and security.
- Collaborate with data scientists, analysts, and domain experts to understand data requirements and deliver high‑quality data assets.
- Monitor pipeline performance, troubleshoot issues, and apply best practices for data governance and compliance.
Requirements
- Proven experience with Python and SQL for data manipulation and automation.
- Hands‑on expertise in Azure Data Factory and Snowflake (or comparable cloud data platforms).
- Strong background in designing and implementing ETL processes and data modelling techniques.
- Understanding of healthcare data standards, privacy regulations, and data security best practices.
- Excellent problem‑solving skills and ability to work collaboratively in a multidisciplinary team.