onsite
Databricks Subject Matter Expert - System One
Software Engineer
Lead technical authority on the Databricks Lakehouse Platform, designing end‑to‑end data solutions, optimizing Spark workloads, and guiding clients through complex data transformation projects.
About the role
Key Responsibilities
- Architect and implement scalable Lakehouse solutions on Databricks for enterprise clients.
- Design, develop, and optimize Apache Spark pipelines using Python and SQL.
- Provide hands‑on guidance and mentorship to data engineering teams throughout project lifecycles.
- Collaborate with stakeholders to translate business requirements into robust data models and transformation workflows.
- Establish best practices for performance tuning, security, and cost management on cloud platforms (AWS/Azure).
Requirements
- 10+ years of experience in data engineering or analytics, with at least 5 years focused on Databricks and Spark.
- Deep expertise in Lakehouse architecture, data modeling, and ETL/ELT processes.
- Proficiency in Python, SQL, and cloud services (AWS or Azure).
- Proven track record delivering complex data transformation projects for large enterprises.
- Strong communication skills and ability to mentor junior engineers.
Skills
databricksapache sparksqlpythonaws