remote
Lead Data Engineer - Delphic Global
Data Engineer
Lead Data Engineer with 8+ years of experience designing and optimizing scalable data pipelines on Azure, leveraging ADF, SQL, PySpark, and Microsoft Fabric Lakehouse solutions to deliver high‑performance data warehousing and analytics services.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines using Azure Data Factory, ensuring scalability, reliability, and performance.
- Write and optimize complex SQL queries (T‑SQL/PL‑SQL) for data extraction, transformation, and loading across Azure SQL, Synapse, and ADLS Gen2.
- Implement Spark workloads in PySpark on Databricks or Fabric Spark, applying best practices for performance tuning and resource management.
- Architect and manage Lakehouse solutions in Microsoft Fabric, including Dataflows, Pipelines, and OneLake, to support modern analytics workloads.
- Apply data warehousing concepts such as SCD, star schema, and dimensional modeling to design robust data marts.
- Collaborate with DevOps teams to build CI/CD pipelines in Azure DevOps, automating deployment, testing, and monitoring of data services.
Requirements
- 8+ years of professional experience in data engineering with a strong focus on Azure ecosystem.
- Proficiency in Azure Data Factory, SQL, and PySpark, with hands‑on experience in Microsoft Fabric Lakehouse.
- Deep understanding of data warehousing principles, including star schema design and slowly changing dimensions.
- Experience with Azure Data Services such as ADLS Gen2, Azure SQL, and Synapse Analytics.
- Solid knowledge of CI/CD practices and tools (Azure DevOps, Git) for data pipeline automation.