remote
Databricks Solution Architect
Solutions Architect
Lead design and implementation of scalable data pipelines using Databricks on Azure and AWS, optimizing storage with ADLS, and automating deployments through CI/CD pipelines. Focus on autoscaling and performance tuning for enterprise analytics.
About the role
Key Responsibilities
- Architect end‑to‑end data solutions on Databricks, integrating Azure Data Lake Storage and AWS services to meet business analytics needs.
- Design and implement autoscaling strategies to ensure cost‑effective, high‑performance workloads across cloud platforms.
- Develop and maintain CI/CD pipelines for data workflows, ensuring rapid, reliable deployment and version control.
- Collaborate with data engineering, data science, and DevOps teams to translate business requirements into scalable technical solutions.
- Provide technical guidance and best‑practice recommendations for cloud data architecture and security.
Requirements
- Proven experience as a Solution Architect or senior data engineer with Databricks, Azure, and AWS.
- Deep knowledge of Azure Data Lake Storage, S3, and cloud autoscaling mechanisms.
- Strong background in CI/CD tooling (Git, Azure DevOps, Jenkins, or similar) for data pipelines.
- Excellent problem‑solving skills and ability to communicate complex concepts to technical and non‑technical stakeholders.
- Hands‑on experience with Python or Scala for data processing is a plus.