remote
Senior Data Consultant - IO Associates
Software Engineer
Senior Data Consultant leading end‑to‑end data engineering projects on Databricks, crafting scalable, maintainable pipelines with Python, Spark, and SQL, while ensuring high quality and operational excellence.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines on Databricks, ensuring scalability and performance.
- Implement data modeling, ETL processes, and data quality checks using Python, Spark, and SQL.
- Collaborate with cross‑functional teams to translate business requirements into technical solutions.
- Apply software engineering best practices—version control, CI/CD, testing—to data workflows.
- Mentor junior engineers and promote data engineering standards across projects.
Requirements
- 5+ years of data engineering experience with a strong focus on software engineering principles.
- Proficiency in Databricks, Python, Spark, and SQL.
- Hands‑on experience building production‑grade data pipelines and data models.
- Excellent problem‑solving skills and a passion for clean, maintainable code.
- Strong communication skills and ability to work collaboratively in a fast‑paced environment.
Skills
databrickspythonsql