onsite
Data Engineer II - Walmart
Data Engineer
Senior Data Engineer focused on designing, building, and maintaining scalable data pipelines and foundational data assets using Python, Spark, and AWS to empower advanced analytics, machine learning, and business intelligence.
About the role
Key Responsibilities
- Design, develop, and maintain robust data pipelines that ingest, transform, and load data from diverse sources into enterprise data warehouses.
- Collaborate with data scientists, analysts, and product teams to define data requirements, quality standards, and performance metrics.
- Implement and optimize ETL processes using Python, SQL, and Apache Spark on AWS services such as S3, Redshift, and Glue.
- Ensure data reliability, consistency, and security through rigorous testing, monitoring, and documentation.
- Participate in data architecture reviews, contribute to best‑practice guidelines, and mentor junior engineers.
Requirements
- 3+ years of experience in data engineering, with a strong background in Python, SQL, and Spark.
- Hands‑on experience with AWS data services (S3, Redshift, Glue, Athena).
- Proficiency in data modeling, ETL design, and data warehousing concepts.
- Strong problem‑solving skills and ability to work in a fast‑paced, collaborative environment.
- Excellent communication skills and a passion for clean, maintainable code.
Skills
pythonsqlapache sparkaws