remote
Senior Data Engineer - 2U
Data Engineer
Lead the design and implementation of scalable data pipelines and lakehouse architecture using Python, Spark, and AWS services to support high‑volume analytics for a global online education platform.
About the role
Key Responsibilities
- Architect, develop, and maintain end‑to‑end data pipelines that ingest, transform, and load large volumes of structured and unstructured data into a Snowflake‑based data lakehouse.
- Collaborate with data scientists, product managers, and business stakeholders to define data requirements, data models, and performance metrics.
- Optimize query performance and storage costs through partitioning, clustering, and materialized views in Snowflake and AWS Redshift.
- Implement robust data quality checks, monitoring, and alerting using Airflow, dbt, and CloudWatch.
- Mentor junior engineers, conduct code reviews, and promote best practices in data engineering and DevOps.
Requirements
- 5+ years of experience in data engineering with a strong background in Python, SQL, and Spark.
- Hands‑on experience building data pipelines on AWS (S3, Glue, EMR, Lambda) and deploying them to Snowflake.
- Proficiency in designing data models, ETL processes, and data lake architecture.
- Strong analytical skills and ability to troubleshoot complex data issues.
- Excellent communication skills and a collaborative mindset.
Skills
pythonsqlapache sparkawssnowflake