remote
Azure Data Engineer - Dev Information Technology Limted
Data Engineer
Lead the design and implementation of scalable Azure data platforms, building high‑performance ETL/ELT pipelines with PySpark and Python while ensuring cost efficiency, security, and governance across the data ecosystem.
About the role
Key Responsibilities
- Architect and build scalable data platforms using Azure services such as ADF, Databricks, Synapse, Data Lake, and Azure SQL.
- Design, develop, and optimize large‑scale ETL/ELT pipelines with PySpark and Python, ensuring high performance and reliability.
- Implement CI/CD pipelines and DevOps practices for data workflows, automating deployments and monitoring.
- Define and enforce data architecture standards, governance policies, and best practices across the organization.
- Lead troubleshooting, performance tuning, and cloud migration initiatives to improve data processing efficiency.
- Collaborate with stakeholders, mentor junior engineers, and drive continuous improvement in data engineering processes.
Requirements
- Proven experience with Azure data services: ADF, Databricks, Synapse, Data Lake, and Azure SQL.
- Strong proficiency in PySpark and Python for distributed data processing.
- Hands‑on experience with CI/CD, DevOps tooling, and automated data pipeline deployment.
- Deep understanding of data architecture, governance, security, and cost optimization in cloud environments.
- Excellent problem‑solving skills and ability to mentor and collaborate with cross‑functional teams.