onsite
Senior Data Engineer - FGF Brands
Data Engineer
Senior Data Engineer who designs and delivers scalable real‑time streaming and batch pipelines on Azure Databricks, applying Lakehouse patterns, data modeling, and CI/CD to enable analytics and machine‑learning solutions.
About the role
Key Responsibilities
- Architect, build, and maintain end‑to‑end data pipelines (streaming and batch) on Azure Databricks using Apache Spark.
- Design Lakehouse‑style data models that support analytics, reporting, and machine‑learning workloads.
- Implement CI/CD practices for data workflows, ensuring reliable deployments and version control.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into scalable data solutions.
- Monitor performance, troubleshoot issues, and optimize data processing for cost and speed.
Requirements
- 5+ years of professional data engineering experience, with a focus on Azure Databricks or similar cloud data platforms.
- Strong proficiency in Python and SQL for data transformation and analysis.
- Deep knowledge of Apache Spark, Lakehouse architecture, and data modeling best practices.
- Hands‑on experience implementing CI/CD pipelines for data engineering (e.g., Azure DevOps, GitHub Actions).
- Ability to work autonomously in a fast‑paced environment and mentor junior engineers.
Skills
apache sparkpythonsqlcicd