remote
Senior Data Engineer - IO Associates
Data Engineer
Senior Data Engineer needed to design, build, and maintain production‑grade data pipelines and platforms using Python, SQL, Airflow, Snowflake, Kafka and AWS for a high‑impact, remote contract.
About the role
Key Responsibilities
- Design, develop, and operate scalable, production‑grade data pipelines that support core platform modernization initiatives.
- Implement robust ETL/ELT processes using Apache Airflow, ensuring reliability, monitoring, and alerting.
- Build and optimise data models and warehouses in Snowflake, delivering performant, query‑ready datasets for downstream applications.
- Integrate real‑time streaming data via Kafka, handling ingestion, transformation, and delivery to analytical stores.
- Collaborate closely with software engineers, product owners, and architects to embed data solutions within broader software systems.
- Apply best practices for data quality, security, and governance while operating in a cloud environment (AWS).
Requirements
- 5+ years of hands‑on experience building and maintaining data pipelines in production.
- Strong proficiency in Python and SQL for data manipulation and automation.
- Deep experience with Apache Airflow, Snowflake, and Kafka or comparable orchestration/streaming technologies.
- Solid understanding of cloud platforms, preferably AWS, including services such as S3, Redshift, and IAM.
- Demonstrated ability to work autonomously in a remote setting and collaborate with cross‑functional engineering teams.
Skills
pythonsqlsnowflakekafkaaws