remote
Senior Data Engineer - Melonleaf Consulting
Data Engineer
Lead the design and deployment of a Databricks Lakehouse for CRM data, building scalable ETL/ELT pipelines from Salesforce and other sources, while ensuring GDPR compliance with Unity Catalog and IaC via Terraform.
About the role
Key Responsibilities
- Architect and implement a greenfield Databricks Lakehouse for the CRM domain, integrating Salesforce, Redshift, and data lake sources.
- Design end‑to‑end ETL/ELT pipelines that transform raw data into a Medallion architecture, ensuring high quality and performance.
- Serve as the technical design authority, guiding best practices for data modeling, governance, and security.
- Apply Infrastructure as Code with Terraform to provision, configure, and secure the Databricks workspace and Unity Catalog.
- Enforce GDPR and PII compliance through catalog policies, data masking, and audit logging.
- Collaborate with data scientists to build ML infrastructure that supports advanced customer forecasting and strategic decision‑making.
Requirements
- 5+ years of experience as a Data Engineer, with deep expertise in Databricks and Lakehouse concepts.
- Proven track record designing and deploying ETL/ELT pipelines from Salesforce and other enterprise data sources.
- Strong proficiency in Terraform for cloud infrastructure automation.
- Hands‑on experience with Unity Catalog, data governance, and GDPR/PII compliance.
- Excellent communication skills and ability to lead cross‑functional technical teams.
Skills
databrickssalesforceterraform