remote
Senior Data Engineer - Nbi Consulting
Data Engineer
Senior Data Engineer responsible for designing and operating a real‑time telemetry pipeline that ingests millions of game events, processes them with Spark/Kafka, and delivers analytical datasets on AWS for designers and leadership.
About the role
Key Responsibilities
- Design, build, and maintain end‑to‑end telemetry pipelines that collect event data from game clients and servers.
- Develop real‑time streaming solutions using Apache Kafka and Apache Spark to process high‑volume player events.
- Create and manage scalable data warehouses on AWS (Redshift, S3, Glue) to store raw and transformed datasets.
- Implement robust ETL workflows in Python/SQL, ensuring data quality, reliability, and low latency.
- Collaborate with game designers, producers, and analysts to deliver actionable data models and dashboards.
- Monitor, troubleshoot, and optimize pipeline performance and cost efficiency.
Requirements
- 5+ years of experience building large‑scale data pipelines and warehouses.
- Strong proficiency in Python, SQL, and cloud services (AWS).
- Hands‑on experience with Apache Kafka, Apache Spark, and streaming architectures.
- Solid understanding of ETL best practices, data modeling, and performance tuning.
- Excellent communication skills and ability to work cross‑functionally with engineering and game design teams.
Skills
pythonsqlapache sparkaws