onsite
Tech Lead - Databricks
Software Engineer
Lead a cross‑functional team delivering data solutions on Databricks, leveraging AWS, Azure, and Azure Data Factory while driving Agile practices and ensuring high‑performance analytics pipelines.
About the role
Key Responsibilities
- Architect, develop, and maintain scalable data pipelines on Databricks using Spark and Python.
- Design and implement end‑to‑end data solutions across AWS and Azure environments, including Azure Data Factory orchestration.
- Lead a team of engineers, fostering Agile ceremonies, code reviews, and continuous improvement.
- Collaborate with product owners and stakeholders to translate business requirements into technical specifications.
- Ensure data quality, security, and performance monitoring throughout the data lifecycle.
Requirements
- 5+ years of hands‑on experience with Databricks, Spark, and Python for data engineering.
- Strong background in cloud platforms, specifically AWS and Azure services.
- Proven experience with Azure Data Factory or similar orchestration tools.
- Demonstrated ability to lead technical teams in an Agile environment.
- Excellent problem‑solving skills and ability to communicate complex concepts to both technical and non‑technical audiences.
Skills
databricksawsazureagilepython