remote
Data Engineer - WestJet Airlines
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines and lakehouse solutions using Python, SQL, Airflow, Spark, and AWS services to support airline operations analytics.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines for ingesting, transforming, and loading operational data into data lakehouse environments.
- Implement and manage Airflow DAGs to orchestrate ETL workflows, ensuring reliability, scalability, and performance.
- Collaborate with data scientists and business analysts to define data models, schemas, and metadata standards that support analytics and reporting.
- Optimize Spark jobs and SQL queries for large‑scale data processing, monitoring resource usage and tuning performance.
- Leverage AWS services (S3, Glue, Redshift, Athena) to build secure, cost‑effective data infrastructure.
- Document data pipelines, data dictionaries, and best practices; participate in code reviews and continuous improvement initiatives.
Requirements
- 5+ years of experience as a Data Engineer in a data‑intensive environment.
- Proficiency in Python, SQL, and Spark for data processing and transformation.
- Hands‑on experience with Airflow for workflow orchestration.
- Strong knowledge of AWS data services (S3, Glue, Redshift, Athena).
- Solid understanding of data modeling, ETL design, and data governance principles.
Skills
pythonsqlairflowaws