onsite
Senior Data Engineer - Bionic Group
Data Engineer
Senior Data Engineer leading end‑to‑end data pipeline design and implementation using Python, SQL, Spark, Airflow on AWS, to deliver reliable, scalable analytics for a high‑traffic business comparison platform.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load large volumes of business‑critical data from multiple external partners.
- Leverage AWS services (S3, Redshift, Glue, Lambda) to build scalable, cost‑effective data infrastructure.
- Implement Spark jobs and Airflow DAGs to orchestrate complex ETL workflows, ensuring data quality and performance.
- Collaborate with data scientists, product managers, and business stakeholders to translate analytical requirements into technical solutions.
- Monitor pipeline health, troubleshoot issues, and continuously optimize for speed and reliability.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python and SQL.
- Hands‑on expertise with Spark, Airflow, and AWS data services.
- Proven ability to design scalable, maintainable data architectures in a fast‑moving environment.
- Excellent problem‑solving skills and a collaborative mindset.
- Experience with data quality frameworks and monitoring tools is a plus.
Skills
pythonsqlawsairflow