remote
Databricks Platform SME - Tech3pillars Technologies
Software Engineer
Senior Databricks Platform SME overseeing Azure/AWS deployments, security, migrations, and new tool integrations. Expert in IaC, platform provisioning, and administration to ensure high availability and compliance.
About the role
Key Responsibilities
- Lead the design, deployment, and maintenance of Databricks environments on Azure and AWS.
- Implement Infrastructure as Code (IaC) using Terraform or ARM templates for consistent, repeatable provisioning.
- Ensure platform security, compliance, and governance through role-based access, network policies, and audit logging.
- Drive migration projects, including data, workloads, and notebooks, minimizing downtime and data loss.
- Integrate new NPI tools and third‑party services, evaluating performance and cost impact.
- Collaborate with data engineering, analytics, and DevOps teams to optimize cluster usage and cost.
Requirements
- 5+ years of experience managing Databricks on Azure and AWS.
- Proven expertise in IaC, security best practices, and platform governance.
- Strong scripting skills (Python, Bash) and familiarity with CI/CD pipelines.
- Excellent communication and stakeholder management abilities.
- Relevant certifications (e.g., Databricks Certified Professional, Azure/AWS Cloud Architect) are a plus.