onsite
Senior Data Engineer - Azure Databricks / Lakehouse - Deichmann SE
Data Engineer
Lead the design, implementation, and maintenance of scalable data pipelines on Azure Databricks, building a Lakehouse architecture that supports advanced analytics and machine learning workloads.
About the role
Key Responsibilities
- Design, develop, and optimize data pipelines using Azure Databricks and Spark to ingest, transform, and store data in a Lakehouse environment.
- Collaborate with data scientists and analysts to create reproducible data models and support ML workflows.
- Implement data quality, governance, and security controls across the data platform.
- Monitor pipeline performance, troubleshoot issues, and continuously improve throughput and reliability.
- Document architecture, processes, and best practices for internal teams.
Requirements
- 5+ years of experience in data engineering with a focus on Azure services.
- Proficient in Python, Spark, and SQL for large‑scale data processing.
- Hands‑on experience building Lakehouse solutions on Azure Databricks.
- Strong understanding of data modeling, ETL design, and data governance.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonapache sparksql