onsite
Senior Data Engineering Architect - Real Time Intelligence Expert - Avanade
Software Engineer
Lead architect designing real‑time data pipelines and analytics solutions on AWS, leveraging Spark, Kafka, and data lake technologies to deliver high‑value insights and self‑service reporting for enterprise clients.
About the role
Key Responsibilities
- Architect and oversee end‑to‑end real‑time data pipelines using Spark, Kafka, and AWS services.
- Design scalable data lake and warehouse solutions that support self‑service analytics and reporting.
- Lead technical delivery on high‑risk, complex projects, ensuring quality and on‑time delivery.
- Collaborate with stakeholders to translate business requirements into robust data architectures.
- Mentor and guide junior engineers, fostering best practices in data engineering and DevOps.
Requirements
- 10+ years of experience in data engineering, with a focus on real‑time streaming and batch processing.
- Proficiency in Python, Apache Spark, and Apache Kafka.
- Deep knowledge of AWS data services (Glue, Redshift, S3, Kinesis).
- Strong SQL skills and experience with data lake and warehouse design.
- Excellent communication and stakeholder management abilities.
Skills
pythonapache sparkawssql