onsite
Hadoop Big Data Developer - Bright Vision Technologies
Software Engineer
Develop and maintain large‑scale data pipelines on Hadoop ecosystems, leveraging Spark, Java, Hive, and Python to process, transform, and analyze massive datasets for business automation and optimization.
About the role
Key Responsibilities
- Design, develop, and optimize end‑to‑end data pipelines using Hadoop, Spark, and MapReduce.
- Implement data ingestion, transformation, and storage solutions with Hive, Pig, and HDFS.
- Write high‑performance Java and Python code for batch and streaming processing.
- Collaborate with data scientists and analysts to provide clean, reliable data for analytics and machine‑learning models.
- Monitor, troubleshoot, and tune cluster performance to ensure scalability and reliability.
Requirements
- 3+ years of hands‑on experience with Hadoop ecosystem components (HDFS, YARN, Hive, Pig).
- Proficiency in Java and Python for big‑data application development.
- Experience with Apache Spark (core, SQL, Streaming) and MapReduce programming.
- Strong SQL skills and familiarity with data modeling in relational and NoSQL stores.
- Ability to work in an Agile environment, communicate technical concepts clearly, and solve complex data challenges.
Skills
hadoopapache sparkjavahivepython