remote
Data Engineer, Pricing - Lyft
Data Engineer
Data Engineer focused on building scalable, real‑time data pipelines that power dynamic pricing models, leveraging Python, Spark, Airflow, and AWS to ingest, transform, and serve high‑volume ride‑data for demand forecasting and revenue optimization.
About the role
Key Responsibilities
- Design, develop, and maintain end‑to‑end data pipelines that ingest ride, market, and promotion data at scale.
- Implement robust ETL processes using Python, Spark, and Airflow to transform raw data into analytics‑ready datasets.
- Collaborate with data scientists and product teams to translate pricing models into production‑ready data flows.
- Optimize pipeline performance and reliability on AWS infrastructure, ensuring low latency and high availability.
- Document data schemas, pipeline logic, and operational procedures for cross‑functional teams.
Requirements
- 3+ years of experience building data pipelines in a production environment.
- Strong proficiency in Python, SQL, and Apache Spark.
- Hands‑on experience with Airflow and AWS services (S3, Redshift, EMR).
- Solid understanding of data modeling, data quality, and performance tuning.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlapache sparkairflowaws