remote
Data Engineering Lead - Retail Data Platforms Azure & Databricks - UST
Software Engineer
Lead the design and delivery of enterprise‑scale retail data platforms on Azure and Databricks, driving high‑performance pipelines, data modeling, and cloud architecture while mentoring a team of engineers.
About the role
Key Responsibilities
- Architect, build, and optimize end‑to‑end data pipelines on Azure Databricks using Spark and Python.
- Define and enforce data modeling standards, data governance, and best practices for a retail‑focused data platform.
- Collaborate with product, analytics, and business stakeholders to translate requirements into scalable data solutions.
- Lead, mentor, and grow a team of data engineers, fostering a culture of code quality, automation, and continuous improvement.
- Monitor performance, ensure reliability, and implement cost‑effective cloud resource strategies.
Requirements
- 5+ years of hands‑on data engineering experience, with at least 2 years leading teams.
- Deep expertise in Azure services (Data Lake, Synapse, Data Factory) and Databricks/Spark ecosystem.
- Proficient in Python, SQL, and data modeling techniques for large‑scale retail data.
- Strong understanding of cloud architecture, ETL design patterns, and CI/CD for data pipelines.
- Excellent communication skills and proven ability to work cross‑functionally in a fast‑paced environment.
Skills
azuredatabricksapache sparkpythonsql