onsite
Data Engineer Tech Lead - AWS - Advent Global Solutions, Inc.
Data Engineer
Lead the design and implementation of scalable batch and streaming data pipelines on AWS, leveraging Java, Python, and PySpark to deliver high‑quality datasets for analytics and machine learning.
About the role
Key Responsibilities
- Architect and develop robust batch and streaming data pipelines using Java, Python, and PySpark on AWS services (Glue, EMR, Kinesis, S3).
- Design event‑driven data flows and implement real‑time processing with AWS Lambda, Step Functions, and EventBridge.
- Optimize cloud data platforms for performance, cost, and reliability, including data lake and warehouse solutions.
- Collaborate with data scientists and analysts to ensure data quality, lineage, and accessibility for analytics, reporting, and ML workloads.
- Mentor and lead a small team of engineers, driving best practices in coding, testing, and CI/CD.
Requirements
- 15+ years of experience in data engineering and software development.
- Deep expertise in Java, Python, and PySpark for large‑scale data processing.
- Proven track record building event‑driven architectures on AWS.
- Strong understanding of data lake, data warehouse, and analytics best practices.
- Excellent communication skills and ability to mentor junior engineers.