remote
Senior Principal Data Engineer - Mastercard
Data Engineer
Lead data engineering initiatives at scale, designing and building robust data pipelines and platforms using Python, Spark, SQL, and AWS to enable real‑time analytics and insights.
About the role
Key Responsibilities
- Architect, develop, and maintain high‑performance data pipelines and lake/warehouse solutions on AWS.
- Lead the design of data models and schemas to support analytics, machine learning, and reporting use cases.
- Collaborate with cross‑functional teams to define data requirements, ensure data quality, and drive data governance.
- Mentor and guide senior engineers, fostering best practices in code quality, testing, and CI/CD for data workloads.
- Evaluate and integrate emerging technologies such as streaming platforms (Kafka) and distributed processing frameworks (Spark) to enhance platform capabilities.
Requirements
- 10+ years of hands‑on experience in data engineering, with a strong focus on building scalable pipelines in cloud environments.
- Expertise in Python, SQL, and Apache Spark for batch and real‑time processing.
- Deep knowledge of AWS services (e.g., S3, Redshift, Glue, EMR, Lambda) and infrastructure‑as‑code practices.
- Proven ability to design robust data models and implement data governance and quality frameworks.
- Strong leadership and communication skills, with experience mentoring senior technical staff.
Skills
pythonsqlapache sparkawskafka