onsite
Databricks Engineer - Guidehouse Careers
Software Engineer
Senior Databricks Engineer driving large‑scale data pipelines, troubleshooting production workloads, and leading Agile delivery for DHS projects using Spark, Python, SQL, and AWS services.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Apache Spark and Python.
- Diagnose and resolve production issues, ensuring high availability and performance of data workloads.
- Collaborate with cross‑functional Agile teams to deliver data solutions that meet DHS mission objectives.
- Implement best practices for data governance, security, and compliance within the public sector environment.
- Document architecture, code, and operational procedures for knowledge transfer and audit readiness.
Requirements
- 5+ years of experience in data engineering with a focus on Databricks and Spark.
- Proficiency in Python, SQL, and cloud platforms (AWS preferred).
- Strong understanding of data governance, security, and compliance in government contexts.
- Excellent communication skills and ability to work in Agile teams.
- Valid security clearance or ability to obtain Public Trust clearance.
Skills
databricksapache sparkpythonsqlaws