onsite
Azure Data Engineering Data Lake & Streaming - Cognizant
Software Engineer
Azure Data Engineer focused on designing and building scalable Data Lake and streaming solutions using Spark, Kafka, and modern Azure data frameworks to enable high‑performance, data‑driven decision making.
About the role
Key Responsibilities
- Design, develop, and maintain scalable Azure Data Lake architectures for structured and unstructured data across multiple business domains.
- Build and optimize batch and real‑time data pipelines using Spark, Kafka, and Azure Data Factory.
- Collaborate with architects, product owners, and cross‑functional teams to translate business requirements into robust data solutions.
- Implement data quality, governance, and security best practices within the data platform.
- Monitor, troubleshoot, and continuously improve pipeline performance and reliability.
Requirements
- Proven experience with Azure Data Lake, Spark, and Kafka in a production environment.
- Strong knowledge of data engineering principles, ETL/ELT processes, and data modeling.
- Hands‑on experience with Azure Data Factory, Azure Databricks, and related Azure services.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
- Effective communication skills for interacting with stakeholders and technical teams.