onsite
Databricks Specialist - eTelligent Group LLC
Software Engineer
Lead data engineering initiatives using Databricks and Apache Spark, building scalable pipelines and analytics solutions in a cloud environment. Drive performance, data quality, and collaboration across federal data projects.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Spark, Python, and SQL.
- Optimize performance of batch and streaming workloads, ensuring high throughput and low latency.
- Collaborate with data scientists and analysts to transform raw data into actionable insights.
- Implement data governance, security, and compliance best practices in a federal environment.
- Monitor, troubleshoot, and document pipeline health and performance metrics.
Requirements
- 5+ years of experience in data engineering with a focus on Databricks and Spark.
Skills
databricksapache sparkpythonsql