onsite
Azure Data Engineer - ISite Technologies Inc
Data Engineer
Design, build, and maintain scalable data pipelines and lakehouse solutions on Azure using Databricks, collaborating with analysts to deliver reliable, high‑quality data products.
About the role
Key Responsibilities
- Design and implement end‑to‑end data pipelines on Azure Databricks, following Medallion architecture (Bronze, Silver, Gold).
- Develop and optimize Spark jobs using Python/SQL to ingest, transform, and load data from diverse sources.
- Collaborate with business analysts and stakeholders to translate requirements into robust data models and datasets.
- Monitor and tune platform performance, ensuring data quality, integrity, and security throughout the data lifecycle.
- Implement CI/CD practices for data workflows and maintain documentation for data assets.
Requirements
- Strong experience with Azure services (Data Lake Storage, Synapse, ADLS) and Databricks.
- Proficiency in Python and SQL for data processing and analytics.
- Hands‑on expertise with Apache Spark for large‑scale data transformations.
- Understanding of data governance, security, and best practices for data lakehouse solutions.
- Ability to work cross‑functionally, communicate technical concepts, and troubleshoot production issues.
Skills
azuredatabrickspythonsqlapache spark