remote
Hadoop Developer - BV Teck
Software Engineer
Seeking a Hadoop Developer to design, build, and maintain large-scale data pipelines using Hadoop ecosystem tools such as Spark, Hive, and MapReduce, ensuring high performance and reliability in a fully remote environment.
About the role
Key Responsibilities
- Design, develop, and optimize Hadoop-based data pipelines for ingestion, transformation, and analytics.
- Implement Spark jobs and Hive queries to process terabyte‑scale datasets efficiently.
- Maintain and troubleshoot HDFS, YARN, and related cluster components for stability and performance.
- Collaborate with data scientists and analysts to translate business requirements into scalable data solutions.
- Document architecture, code, and best practices for future maintenance and knowledge transfer.
Requirements
- 3+ years of experience with Hadoop ecosystem, including HDFS, YARN, MapReduce, Spark, and Hive.
- Strong proficiency in Java and/or Scala for writing distributed processing code.
- Hands‑on experience with data modeling, ETL design, and performance tuning.
- Familiarity with cloud storage and compute services (AWS, Azure, or GCP) is a plus.
- Excellent problem‑solving skills and ability to work independently in a remote setting.