remoteonsite
Data Engineer - Innoventes Technologies
Data Engineer
Senior Data Engineer building scalable batch and real‑time pipelines on cloud platforms, optimizing ETL/ELT workflows, and ensuring data quality and observability for ML and analytics teams.
About the role
Key Responsibilities
- Design, build, and maintain scalable batch and real‑time data pipelines using Apache Spark, Kafka, Flink, and Airflow.
- Develop and optimize ETL/ELT workflows to ingest data from APIs, databases, event streams, and flat files.
- Architect and manage cloud‑based data infrastructure on AWS, GCP, or Azure, leveraging services such as S3, BigQuery, Redshift, Databricks, and Snowflake.
- Implement data quality monitoring, alerting, and observability frameworks to ensure pipeline reliability and SLA compliance.
- Collaborate with data scientists and ML engineers to support model training, feature engineering, and inference pipelines.
- Partner with analytics engineering teams to deliver high‑quality data products and dashboards.
Requirements
- 5+ years of experience in data engineering and pipeline development.
- Hands‑on experience with cloud data platforms (AWS, GCP, or Azure) and data warehouses (Snowflake, Redshift, BigQuery).
- Solid understanding of data quality, monitoring, and observability best practices.
- Excellent communication skills and ability to work cross‑functionally with data science and analytics teams.
Skills
apache sparkkafkaairflowawssnowflake