onsite
Founding Data Engineer - Dex
Data Engineer
Lead the design and implementation of scalable data pipelines and analytics infrastructure using Python, Spark, and AWS services, driving data-driven decision making for a mission‑driven talent platform.
About the role
Key Responsibilities
- Architect, build, and maintain end‑to‑end data pipelines that ingest, transform, and store large volumes of structured and unstructured data.
- Collaborate with data scientists and product teams to deliver high‑quality datasets for machine learning and business intelligence.
- Implement robust data quality, monitoring, and alerting using Airflow and custom metrics.
- Optimize performance and cost of data workloads on AWS (S3, Redshift, EMR, Glue).
- Drive continuous improvement of data architecture, documentation, and best practices.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python and SQL.
- Hands‑on expertise with Apache Spark and distributed data processing.
- Proficiency in AWS data services (S3, Redshift, EMR, Glue, Athena).
- Experience designing and managing Airflow DAGs for production workloads.
- Excellent problem‑solving skills and a passion for building scalable, reliable data systems.
Skills
pythonsqlapache sparkawsairflow