remote
Technical Expert Databricks - LBBW
Software Engineer
Lead the design, implementation, and optimization of Databricks‑based data platforms, driving advanced analytics and data engineering solutions using Spark, Python, SQL, and Azure services.
About the role
Key Responsibilities
- Architect, develop, and maintain scalable data pipelines on Databricks and Apache Spark.
- Collaborate with data scientists and business analysts to deliver end‑to‑end analytics solutions.
- Implement best practices for data security, governance, and performance tuning in Azure environments.
- Automate deployment and monitoring using CI/CD pipelines and infrastructure‑as‑code tools.
- Provide technical mentorship and guidance to junior engineers and cross‑functional teams.
Requirements
- Strong expertise in Databricks, Apache Spark, and cloud platforms (Azure preferred).
- Proficient programming skills in Python and SQL for data transformation and analysis.
- Hands‑on experience with CI/CD, containerization (Docker/Kubernetes), and infrastructure automation.
- Solid understanding of data engineering concepts, data modeling, and ETL processes.
- Excellent problem‑solving abilities and communication skills to work in an agile, collaborative environment.
Skills
databricksapache sparkpythonsqlazurecicd