onsite
Senior Data Engineer - Amazon
Data Engineer
Senior Data Engineer building enterprise‑scale data pipelines on AWS, leveraging Python, Spark, and SQL to enable data‑driven decisions and AI for customer service.
About the role
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and lakehouse architecture on AWS to ingest, transform, and serve high‑volume customer service data.
- Implement robust ETL workflows using Python, Spark, and SQL, ensuring data quality, lineage, and compliance with privacy regulations.
- Collaborate with data scientists and product teams to expose clean, enriched datasets that power AI models and analytics dashboards.
- Optimize performance and cost of data processing jobs, applying best practices in partitioning, caching, and resource provisioning.
- Monitor pipeline health, troubleshoot failures, and continuously improve reliability and observability.
Requirements
- 5+ years of experience as a data engineer in a large, cloud‑based environment.
- Proficiency in Python, SQL, and Apache Spark for data transformation and orchestration.
- Deep knowledge of AWS services (S3, Glue, Redshift, Athena, EMR, Lambda) and data lake architecture.
- Strong understanding of data modeling, schema design, and ETL best practices.
- Experience with data security, privacy compliance, and performance tuning.
Skills
pythonsqlawsapache spark