remote
Senior Data Engineer Contract - Codurance
Data Engineer
Senior Data Engineer leading large‑scale data pipelines on Databricks, building robust Spark jobs in Python, optimizing SQL queries, and managing Delta Lake storage to deliver reliable, production‑ready analytics solutions.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines on Databricks using Apache Spark and Python.
- Implement and optimize SQL queries for data extraction, transformation, and loading across enterprise data lakes.
- Ensure data quality, lineage, and governance through Delta Lake best practices and automated testing.
- Collaborate with data scientists, analysts, and product teams to translate business requirements into technical solutions.
- Monitor pipeline performance, troubleshoot issues, and continuously improve throughput and reliability.
Requirements
- 5+ years of experience in data engineering with a strong focus on Spark and Databricks.
- Proficient in Python, SQL, and Delta Lake architecture.
- Hands‑on experience building production‑grade ETL workflows and data lake solutions.
- Solid understanding of data modeling, performance tuning, and cloud data services.
- Excellent problem‑solving skills and a passion for clean, maintainable code.
Skills
databricksapache sparkpythonsql