onsite
Senior Data Engineer - 42dot
Data Engineer
Lead the design and operation of large‑scale vehicle data pipelines, optimizing batch and streaming architectures, ensuring data quality, and mentoring junior engineers while driving data‑driven insights for mobility services.
About the role
Key Responsibilities
- Design, build, and maintain high‑throughput data pipelines ingesting over 100 million records daily from connected vehicles.
- Architect and optimize both batch and streaming data workflows, ensuring low latency and high reliability.
- Establish automated data quality checks and observability frameworks to monitor pipeline health and performance.
- Lead technical decision‑making within the team and mentor junior engineers on best practices.
- Collaborate with cross‑functional stakeholders to translate business requirements into scalable data solutions.
Requirements
- 8+ years of software development experience, with at least 3 years focused on data engineering.
- Proven track record designing and operating large‑scale data pipelines (100M+ records/day).
- Hands‑on experience with streaming and batch architectures using Python or JVM languages (Java, Scala, Kotlin).
- Strong SQL skills for data modeling, analysis, and performance tuning.
- Experience implementing observability and automated data quality frameworks.
Skills
pythonjavascalakotlinsql