remote
AWS Databricks Engineer - Capgemini
Software Engineer
AWS Databricks Engineer designing, building, and optimizing scalable Lakehouse data pipelines in AWS, leveraging PySpark, Spark SQL, Delta Lake, and Python to deliver high‑performance, secure data solutions for a banking client.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines on Databricks within AWS environments.
- Implement Delta Lake best practices for ACID transactions, schema evolution, and time travel.
- Optimize PySpark and Spark SQL workloads for performance and cost efficiency.
- Collaborate with data scientists and business stakeholders to translate requirements into robust data models.
- Ensure data security, governance, and compliance across all lakehouse components.
Requirements
- 5+ years of experience in data engineering with a focus on AWS and Databricks.
Skills
awsdatabrickspythonsql