remoteonsite
Lead Data Engineer - Databricks - Brillio LLC
Data Engineer
Lead Data Engineer specializing in Databricks, responsible for designing and optimizing large-scale data pipelines, advanced SQL and Python development, and modern data platform architecture.
About the role
Key Responsibilities
- Design, develop, and maintain scalable ETL pipelines on Databricks using SQL and Python.
- Implement data modeling and warehousing solutions to support analytics and reporting.
- Optimize performance of Spark jobs, SQL queries, and stored procedures (PLSQL/T‑SQL).
- Collaborate with data scientists and business stakeholders to translate requirements into robust data solutions.
- Ensure data quality, governance, and security across the data platform.
Requirements
- 5–10 years of experience in data engineering with a focus on Databricks.
- Hands‑on experience with ETL, data warehousing, and data modeling concepts.
- Strong knowledge of modern data platform fundamentals and best practices.
- Excellent problem‑solving skills and ability to work in a collaborative, hybrid environment.
Skills
sqlpythondatabricks