remote
Senior Databricks Platform Engineer - Azure Cloud & Data Operations - Kutir Inc
Devops Engineer
Lead the design, deployment, and optimization of Azure‑native Databricks solutions, driving automation, platform reliability, and data‑operations excellence for a large‑scale enterprise modernization effort.
About the role
Key Responsibilities
- Architect and implement scalable Databricks clusters on Azure, ensuring high availability, performance, and cost efficiency.
- Develop and maintain CI/CD pipelines for data pipelines, notebooks, and infrastructure-as-code using Azure DevOps or GitHub Actions.
- Collaborate with data scientists and engineers to optimize Spark jobs, monitor workloads, and troubleshoot performance bottlenecks.
- Implement security, governance, and compliance controls across the data platform, including role‑based access, encryption, and audit logging.
- Drive automation of data ingestion, transformation, and cataloging processes using Azure Data Factory, Azure Synapse, and Delta Lake.
- Provide mentorship and technical guidance to junior platform engineers and data teams.
Requirements
- 5+ years of experience in data platform engineering with a focus on Databricks and Azure.
- Proficient in Spark, Scala/Java/Python, and Delta Lake architecture.
- Hands‑on experience with Azure services: Data Factory, Synapse, ADLS, Key Vault, and Azure Monitor.
- Strong background in CI/CD, IaC (Terraform/ARM), and container orchestration (AKS).
- Excellent problem‑solving skills, strong communication, and a collaborative mindset.