onsite
Lead AWS Data Engineer - Weekday
Data Engineer
Lead the design and implementation of scalable, cloud‑native data platforms on AWS, building robust ETL pipelines, data models, and analytics solutions using Python, Spark, and Airflow for enterprise‑wide reporting and AI initiatives.
About the role
Key Responsibilities
- Architect, develop, and maintain high‑performance data pipelines on AWS using services such as Redshift, S3, Glue, and EMR.
- Design and implement ETL/ELT workflows with Apache Airflow and Python to ingest, transform, and load data from diverse sources.
- Build scalable processing jobs with Apache Spark and optimize SQL queries for large‑volume analytical workloads.
- Collaborate with data scientists, analysts, and business stakeholders to translate requirements into reliable data models and datasets.
- Ensure data security, governance, and cost‑efficiency through best practices, monitoring, and automated testing.
Requirements
- 7+ years of professional experience in data engineering, with at least 3 years focused on AWS cloud services.
- Strong proficiency in Python, SQL, and Spark for batch and streaming data processing.
- Hands‑on experience building and orchestrating pipelines with Apache Airflow or similar workflow tools.
- Deep understanding of data modeling, warehouse design, and ETL/ELT best practices.
- Proven ability to work cross‑functionally, solve complex data problems, and deliver production‑grade solutions at scale.
Skills
awspythonapache sparksqlairflow