remote
Principal Data Engineer, Data Platform - A Place For Mom
Data Engineer
Lead the design and evolution of a scalable, AI‑driven data platform on Databricks, driving data governance, self‑service analytics, and machine learning product development across the organization.
About the role
Key Responsibilities
- Architect and maintain a unified data lake on Databricks, ensuring high performance, reliability, and security for analytics and ML workloads.
- Define and enforce data governance policies, including cataloging, lineage, and access controls to enable trusted self‑service analytics.
- Collaborate with data scientists, product owners, and business stakeholders to translate business requirements into scalable data pipelines and ML models.
- Mentor and lead a team of data engineers, fostering best practices in code quality, testing, and CI/CD for data workflows.
- Drive continuous improvement of data platform tooling, monitoring, and cost optimization across cloud resources.
Requirements
- 10+ years of experience in data engineering, with a strong background in Spark/Databricks and Python.
- Proven expertise in data lake architecture, data governance frameworks, and metadata management.
- Hands‑on experience building end‑to‑end ML pipelines and deploying models at scale.
- Strong leadership skills, with a track record of mentoring and scaling engineering teams.
- Excellent communication skills and ability to translate complex technical concepts to non‑technical stakeholders.
Skills
databrickspythonapache sparkmachine learningsql