onsite
Data Platform Engineer - Central Garden & Pet
Data Engineer
Design, build, and operate a cloud‑native data platform on Databricks, creating automated CI/CD pipelines for data ingestion, transformation, and analytics while ensuring security, scalability, and reliability.
About the role
Key Responsibilities
- Architect, deploy, and manage the Databricks environment, including workspaces, clusters, job scheduling, and access controls.
- Develop and maintain CI/CD pipelines that automate data ingestion, ETL/ELT processes, and analytics workloads.
- Collaborate with data engineers, analytics engineers, and governance teams to define platform standards, security policies, and performance benchmarks.
- Implement infrastructure‑as‑code using Terraform (or similar) to provision and version‑control cloud resources on Azure.
- Monitor platform health, troubleshoot incidents, and continuously improve reliability and cost efficiency.
Requirements
- Strong experience with Databricks (workspace administration, cluster management, job orchestration).
- Proficiency in Python and SQL for data processing and automation.
- Hands‑on experience building CI/CD pipelines with tools such as Azure DevOps, GitHub Actions, or Jenkins.
- Solid understanding of cloud infrastructure (Azure) and IaC tools like Terraform.
- Knowledge of data security, governance, and best practices for scalable data platforms.
Skills
databrickspythonsqlcicdterraformazure