onsite
Core Big Data Engineer - Hadoop Ecosystem - Zensar Technologies
Data Engineer
Senior Big Data Engineer focused on Hadoop ecosystem, driving high‑performance data pipelines with Spark, Scala, Python, and Airflow orchestration.
About the role
Key Responsibilities
- Design, develop, and maintain large‑scale data pipelines using Hadoop, Spark, and related ecosystem components.
- Implement data ingestion, transformation, and storage solutions with Scala and Python, ensuring optimal performance and scalability.
- Orchestrate complex workflows and schedule jobs using Apache Airflow, monitoring execution and troubleshooting failures.
- Collaborate with data scientists and analysts to translate business requirements into robust data models and ETL processes.
- Optimize cluster resources, tune job configurations, and implement best practices for data quality and security.
Requirements
- 5+ years of experience in big data engineering, with deep knowledge of Hadoop, Spark, and related tools.
- Proficiency in Scala and Python for data processing and automation.
- Hands‑on experience with Apache Airflow for workflow orchestration.
- Strong understanding of distributed computing concepts, data storage architectures, and performance tuning.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
hadoopapache sparkscalapython