remote
Data Engineer - Azure - Tata Consultancy Services (TCS)
Data Engineer
Design, develop, and maintain scalable data pipelines on Azure, leveraging SQL, Python, and Azure Data Factory to enable robust analytics and reporting for enterprise clients.
About the role
Key Responsibilities
- Design and implement end‑to‑end data pipelines on Azure using Azure Data Factory, Azure Databricks, and Spark.
- Develop, test, and optimize SQL queries and Python scripts for data extraction, transformation, and loading (ETL).
- Collaborate with data architects and business analysts to translate requirements into scalable data models.
- Monitor pipeline performance, troubleshoot issues, and ensure data quality and reliability.
- Implement best practices for data security, governance, and cost‑effective cloud resource utilization.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or related field with 3+ years of hands‑on data engineering experience.
- Proficiency in Azure services (Data Factory, Synapse, Databricks) and strong SQL programming skills.
- Experience developing data pipelines in Python and working with Apache Spark or similar big‑data frameworks.
- Solid understanding of ETL concepts, data modeling, and cloud‑based data storage solutions.
- Excellent problem‑solving abilities and effective communication within cross‑functional teams.
Skills
azuresqlpythonapache spark