Data Engineer
Senior Data Engineer responsible for building and scaling a real‑time data platform using Kafka, Spark, and Iceberg on Kubernetes, enabling advanced analytics, AI, and business intelligence in a cloud‑native environment.
Senior Data Engineer (Kafka Streaming, Spark, Iceberg on Kubernetes) Build and scale a next-generation real-time data platform with cutting-edge open-source technologies. 100% Remote | R100 000 – R110 000 per month
About Our Client
Our client is a rapidly growing technology-driven organization building high-performance data platforms to enable advanced analytics, AI, and business intelligence. The team operates at the forefront of real-time data processing and distributed systems, leveraging modern cloud-native infrastructure. They foster a culture of technical excellence, continuous learning, and collaboration across multidisciplinary engineering teams.
The Role: Senior Data Engineer
As a Senior Data Engineer, you will design, build, and optimize next-generation data pipelines and platforms. Youll lead the architecture and implementation of scalable, real-time data solutions using Kafka, Spark, and Apache Iceberg deployed on Kubernetes. This is a hands-on, high-impact role within a forward-thinking data engineering team focused on performance, scalability, and innovation.
Key Responsibilities
5+ years of professional experience in data engineering or software engineering
Design and implement scalable, highly available real-time data pipelines and architectures
Build robust ETL and streaming pipelines using Apache Spark (Scala/Python) and Kafka Connect/Streams
Develop and manage data lakes using Apache Iceberg with schema evolution and time travel capabilities
Deploy and manage distributed data processing services on Kubernetes using containerization best practices
Optimize performance and resource usage across Spark jobs, streaming apps, and Iceberg tables
Define and uphold engineering best practices including testing, code standards, and CI/CD workflows
Mentor junior engineers and contribute to building a high-performing data engineering team
About You
5+ years of experience in data engineering or related software engineering roles
Advanced proficiency with Apache Spark (batch and streaming)
In-depth experience with Apache Kafka (Connect, Streams, or ksqlDB)
Hands-on experience with Apache Iceberg, including table evolution and performance tuning
Skilled in Python (PySpark) or Scala
Experience deploying and managing distributed systems on Kubernetes (Spark Operator is a plus)
Solid understanding of data modeling and data warehousing concepts
Advantageous: Experience with AWS, Azure, or GCP; familiarity with Flink or Trino
Preferred: Bachelors or Masters degree in Computer Science, Engineering, or related field
Originally posted on Himalayas
Posted June 21, 2026