remote
Databricks Data Architect - Deloitte
Software Engineer
Senior technical leader designing and optimizing a scalable data platform on Databricks, leveraging Spark, Delta Lake, and advanced data modeling to deliver high‑performance analytics and data pipelines.
About the role
Key Responsibilities
- Architect and implement end‑to‑end data pipelines on Databricks, ensuring performance, reliability, and scalability.
- Design and maintain Delta Lake tables, schema evolution, and data quality frameworks.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into robust data models and solutions.
- Optimize Spark jobs and SQL queries for cost and speed, applying best practices in partitioning, caching, and resource management.
- Lead code reviews, enforce coding standards, and mentor junior engineers on Databricks and Spark technologies.
Requirements
- 5+ years of experience building data platforms with Databricks and Apache Spark.
- Strong proficiency in Python, SQL, and data modeling concepts.
- Hands‑on experience with Delta Lake, schema evolution, and data quality tooling.
- Deep understanding of ETL/ELT patterns, data lakehouse architecture, and cloud data services (AWS, Azure, or GCP).
- Excellent communication skills and a proven ability to lead technical teams.
Skills
databricksapache sparksqlpython