remote
Data Engineer - Mastercard
Data Engineer
Data Engineer responsible for designing, building, and maintaining scalable data pipelines on AWS, leveraging Python, SQL, and Spark to transform and model data for analytics and machine learning initiatives.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines using Python, SQL, and Apache Spark on AWS services (Glue, Redshift, S3).
- Implement data ingestion, transformation, and quality checks to support analytics, reporting, and ML workloads.
- Collaborate with data scientists, analysts, and product teams to understand data requirements and deliver high‑quality datasets.
- Optimize pipeline performance, monitor job health, and troubleshoot issues in a production environment.
- Document data models, pipeline architecture, and best practices for future reference.
Requirements
- 3+ years of experience in data engineering or related field.
- Experience with data modeling, ETL design, and data quality assurance.
- Excellent problem‑solving skills and ability to work collaboratively in a fast‑paced environment.
Skills
pythonsqlawsapache spark