remote
Senior Principal Databricks Platform Engineering Lead - JPMorganChase
Software Engineer
Lead the design and operation of a Databricks platform on AWS, building Python and Terraform automation, mentoring engineering teams, and driving AI/ML data initiatives to support enterprise analytics.
About the role
Key Responsibilities
- Architect, deploy, and maintain a scalable Databricks environment on AWS, ensuring high availability, security, and cost efficiency.
- Develop and maintain Terraform modules and Python scripts for infrastructure as code, automation, and data pipeline orchestration.
- Mentor and lead cross‑functional engineering teams, fostering best practices in data engineering, DevOps, and cloud architecture.
- Collaborate with data scientists and business stakeholders to design and implement AI/ML solutions that drive actionable insights.
- Define and enforce governance, data quality, and security policies across the data platform.
Requirements
- 10+ years of software engineering experience with a focus on data platforms and cloud infrastructure.
- Deep expertise in Databricks, AWS services (EMR, S3, Glue, IAM), Terraform, and Python.
- Proven track record of leading large‑scale, high‑impact data engineering projects.
- Strong understanding of machine learning workflows, data governance, and security best practices.
- Excellent communication skills and ability to influence stakeholders at all levels.
Skills
databricksawspythonterraformmachine learning