onsite
Senior Data Engineer - Syngenta Seeds
Data Engineer
Senior Data Engineer leading data pipeline design and automation in a collaborative DataOps environment, leveraging Python, SQL, Spark, AWS, Airflow, and Docker to deliver scalable, reliable data solutions for analytics and data science teams.
About the role
Key Responsibilities
- Design, develop, and maintain robust ETL/ELT pipelines using Python, SQL, and Apache Spark to ingest and transform large‑scale agricultural data.
- Implement and manage workflow orchestration with Apache Airflow, ensuring reliable scheduling, monitoring, and error handling.
- Deploy and optimize data infrastructure on AWS services (S3, Redshift, EMR, Lambda), focusing on performance, cost efficiency, and security.
- Containerize applications with Docker and support CI/CD pipelines to streamline development and production releases.
- Collaborate closely with data scientists, analysts, and visualization experts to understand data requirements and deliver clean, well‑documented data sets.
- Establish best practices for data quality, governance, and monitoring, including automated testing and alerting.
Requirements
- 5+ years of professional experience building data pipelines and data platforms in a cloud environment.
- Strong proficiency in Python, SQL, and Apache Spark for batch and streaming workloads.
- Hands‑on experience with AWS services (S3, Redshift, EMR, Lambda) and infrastructure‑as‑code tools.
- Expertise in workflow orchestration using Apache Airflow and containerization with Docker.
- Solid understanding of data modeling, data warehousing concepts, and best practices for data quality and governance.
Skills
pythonsqlapache sparkawsairflowdocker