remote
Senior Data Engineer - Quantiphi
Data Engineer
Senior Data Engineer driving end‑to‑end data pipelines, leveraging Python, SQL, Spark, and AWS to deliver scalable, high‑quality data solutions for analytics and reporting.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines using Python, SQL, and Apache Spark to ingest, transform, and load data from diverse sources into cloud data warehouses.
- Collaborate with data scientists, analysts, and business stakeholders to define data requirements, data models, and performance metrics.
- Implement and optimize ETL processes, ensuring data quality, consistency, and compliance with governance policies.
- Leverage AWS services (S3, Redshift, Glue, Lambda) to build scalable, cost‑effective data architectures.
- Monitor pipeline performance, troubleshoot issues, and continuously improve data workflows.
Requirements
- 5+ years of experience in data engineering, with a strong background in Python, SQL, and Spark.
- Proven expertise in building and maintaining data warehouses on AWS.
- Solid understanding of data modeling, ETL best practices, and data governance.
- Experience with version control (Git) and CI/CD pipelines for data workflows.
- Excellent communication skills and ability to translate business needs into technical solutions.
Skills
pythonsqlapache sparkaws