remote
Senior Manager - Data Engineering - Axis Max Life Insurance
Software Engineer
Lead the design, implementation, and maintenance of scalable data pipelines on AWS, driving data‑driven decision making through robust big data solutions and cross‑functional collaboration.
About the role
Key Responsibilities
- Architect and manage end‑to‑end data pipelines using AWS services (Glue, EMR, Redshift, S3) to support analytics and reporting.
- Develop and optimize Spark jobs in Python for large‑scale data processing and transformation.
- Collaborate with data scientists, product managers, and business stakeholders to translate requirements into scalable data solutions.
- Implement data modeling best practices, ensuring high quality, consistent, and secure data assets.
- Lead performance tuning, monitoring, and troubleshooting of data workflows and infrastructure.
Requirements
- 5+ years of experience in data engineering with a strong focus on AWS cloud architecture.
- Proficiency in Python, SQL, and Apache Spark for ETL and data transformation.
- Hands‑on experience with AWS Glue, EMR, Redshift, and S3, including security and cost optimization.
- Solid understanding of data modeling, schema design, and data governance principles.
- Excellent communication skills and ability to work cross‑functionally in a fast‑paced environment.
Skills
awspythonapache sparksql