remote
Middle Data Engineer Azure Databricks
Data Engineer
Mid-level Data Engineer focused on building and maintaining scalable data pipelines using Azure Databricks, Azure Data Factory, and Apache Spark, ensuring efficient data processing and deployment through Azure DevOps.
About the role
Key Responsibilities
- Design, develop, and optimize data pipelines in Azure Databricks using Apache Spark.
- Integrate data sources with Azure Data Factory for orchestrated ETL workflows.
- Implement CI/CD pipelines with Azure DevOps to automate deployment and testing.
- Collaborate with data scientists and analysts to support data ingestion, transformation, and quality.
- Monitor pipeline performance, troubleshoot issues, and apply best practices for scalability and reliability.
Requirements
- 3+ years of experience in data engineering with Azure services.