onsite
Data Engineer - 42dot
Data Engineer
Data Engineer responsible for building and maintaining end‑to‑end data pipelines, managing schemas, ensuring data quality, and creating dashboards using Python, SQL, Spark, and Databricks in a fast‑moving mobility AI environment.
About the role
Key Responsibilities
- Design, develop, and maintain robust data collection and processing pipelines.
- Manage data schemas and operate a data catalog to ensure consistency across services.
- Implement automated data quality checks and anomaly detection logic.
- Configure dashboards and monitor pipeline metrics for performance and reliability.
- Develop and test ingestion logic to support real‑time and batch data flows.
Requirements
- 3+ years of software development experience, with at least 1 year in data engineering.
- Proficiency in Python or a JVM language (Java, Scala, Kotlin) for data processing.
- Strong SQL skills and experience working with relational databases.
- Version control expertise using Git and participation in code reviews.
- Experience with Apache Spark or Databricks is highly desirable.
Skills
pythonsqlapache sparkdatabricksclickhouse