remote
Software Developer / Data Engineer
Data Engineer
Develop and maintain scalable data pipelines for an EdTech platform using Apache Beam, Flink, and Spark, while designing cloud‑native solutions that support real‑time analytics and robust data processing.
About the role
Key Responsibilities
- Design, implement, and optimize distributed data pipelines using Apache Beam, Flink, and Spark.
- Build cloud‑native services and infrastructure that ensure high availability, scalability, and fault tolerance.
- Collaborate with product and analytics teams to translate educational data requirements into efficient processing workflows.
- Monitor, troubleshoot, and improve performance of streaming and batch jobs in production.
- Implement best practices for data quality, security, and governance across the platform.
Requirements
- Strong experience with Apache Beam, Flink, or Spark in building large‑scale data pipelines.
- Proficiency in cloud‑native technologies and architecture patterns (e.g., containers, Kubernetes, serverless).
- Solid programming skills in languages such as Python, Java, or Scala.
- Understanding of data modeling, ETL processes, and real‑time analytics.
- Ability to work in an agile environment and communicate technical concepts to cross‑functional teams.
Skills
apache beamapache flinkapache spark