onsite
Senior Data Engineer - Cloudflare
Data Engineer
Lead end‑to‑end data pipeline development, architecting scalable solutions on AWS, leveraging Python, Spark, and Airflow to transform and deliver high‑quality data for real‑time analytics and machine learning.
About the role
Key Responsibilities
- Design, build, and maintain large‑scale data pipelines that ingest, transform, and store data from diverse sources.
- Implement robust ETL processes using Python, Spark, and SQL, ensuring data quality and performance.
- Deploy and manage data workflows on AWS services (S3, Redshift, Glue, EMR) and orchestrate with Airflow.
- Collaborate with data scientists and product teams to provide clean, accessible datasets for analytics and ML models.
- Monitor pipeline health, troubleshoot issues, and continuously optimize for cost and speed.
Requirements
- 5+ years of experience in data engineering or related field.
- Strong proficiency in Python, SQL, and Spark for large‑scale data processing.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, EMR) and workflow orchestration (Airflow).
- Deep understanding of data modeling, ETL best practices, and performance tuning.
- Excellent problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
Skills
pythonsqlawsairflow