onsite
Databricks Architect - DXC Technology
Software Engineer
Design and implement Azure‑based Databricks Lakehouse platforms, creating scalable data models, ETL/ELT pipelines, and governance frameworks using Spark, Delta Lake, DBT, Python and SQL.
About the role
Key Responsibilities
- Architect end‑to‑end Databricks Lakehouse solutions on Azure, defining medallion (Bronze/Silver/Gold) data models.
- Develop high‑performance ETL/ELT pipelines with Apache Spark, Delta Lake and DBT, applying schema enforcement, compaction and partitioning strategies.
- Implement data governance, access control, and lineage using Unity Catalog and Azure Purview.
- Optimize Spark workloads for cost and speed through cluster sizing, autoscaling, and job scheduling.
- Design robust data ingestion patterns and ensure reliable data quality across source systems.
Requirements
- Strong experience with Databricks, Azure cloud services, and Apache Spark.
- Advanced proficiency in Python and SQL for data engineering tasks.
- Hands‑on expertise in Delta Lake, DBT, and medallion architecture design.
- Knowledge of data governance tools such as Unity Catalog and Azure Purview.
- Proven ability to design scalable, cost‑effective data pipelines and optimize performance.
Skills
databricksazureapache sparkdbtpythonsql