onsite
RQ11258 - Sr. Software Developer - ETL - Source Code
Software Engineer
Senior Software Developer leading end‑to‑end ETL/ELT pipelines in a Medallion Data Lakehouse environment, leveraging Databricks, Delta Lake, Azure Data Factory, PySpark, dbt, and CI/CD automation to deliver scalable, high‑quality data solutions.
About the role
Key Responsibilities
- Design, develop, and maintain scalable Medallion Data Lakehouse architectures using Delta Lake and Databricks.
- Build, automate, and optimize complex ETL/ELT pipelines with Azure Data Factory, Databricks (PySpark, SQL, Delta Live Tables), and dbt.
- Implement and manage CI/CD pipelines in Azure DevOps and GitHub Actions to ensure rapid, reliable deployments.
- Collaborate with data engineers and data scientists to model and curate data, ensuring high quality and performance.
- Monitor, troubleshoot, and tune pipeline performance, addressing bottlenecks and data quality issues.
Requirements
- 10+ years of experience in data engineering and ETL development.
- Deep expertise in Delta Lake, Databricks, Azure Data Factory, and PySpark.
- Strong knowledge of relational databases (SQL Server, Synapse, PostgreSQL) and dimensional modeling.
- Proven track record of implementing CI/CD for data pipelines using Azure DevOps or GitHub Actions.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.