remote
Deputy Manager - Data Engineer II - Axis Max Life Insurance
Data Engineer
Deputy Manager – Data Engineer II leading scalable data pipelines on AWS, driving big‑data analytics, and collaborating with cross‑functional teams to enable data‑driven decisions.
About the role
Key Responsibilities
- Design, build, and maintain robust data pipelines and infrastructure on AWS to support large‑scale analytics.
- Implement ETL processes using Python, Spark, and other big‑data technologies to transform and load data into data lakes and warehouses.
- Collaborate with data scientists, analysts, and product teams to understand requirements and deliver high‑quality data solutions.
- Optimize pipeline performance, monitor data quality, and troubleshoot issues to ensure reliability and scalability.
- Document architecture, processes, and best practices for future reference and knowledge sharing.
Requirements
- Strong experience with AWS services (S3, Redshift, EMR, Glue, Lambda, Kinesis).
- Proficiency in Python and Spark for data processing and transformation.
- Hands‑on experience building and maintaining ETL pipelines and data lakes.
- Solid understanding of data modeling, SQL, and relational/NoSQL databases.
- Excellent communication skills and ability to work collaboratively in a fast‑paced environment.