remote
Lead Software Engineer - Big Data & AWS - JPMorganChase
Software Engineer
Lead a cross‑functional agile team to design, develop, and deliver scalable big‑data solutions on AWS, leveraging technologies such as Spark, Hadoop, and Kafka while ensuring security, performance, and reliability.
About the role
Key Responsibilities
- Architect, design, and implement high‑performance big‑data pipelines and services on AWS, using Spark, Hadoop, and Kafka.
- Lead an agile development team, providing technical guidance, code reviews, and mentorship to ensure best practices and high‑quality delivery.
- Collaborate with product owners, data scientists, and business stakeholders to translate requirements into robust, scalable solutions.
- Drive performance tuning, capacity planning, and cost‑optimization of cloud resources.
- Ensure security, compliance, and operational stability through automated testing, CI/CD pipelines, and monitoring.
Requirements
- 7+ years of software engineering experience, with at least 3 years leading teams in big‑data or cloud‑native environments.
- Strong proficiency in Java and Python, and hands‑on experience with AWS services (EMR, S3, Lambda, Glue, etc.).
- Deep knowledge of distributed data processing frameworks such as Apache Spark, Hadoop, and streaming platforms like Kafka.
- Solid understanding of relational and NoSQL databases, SQL query optimization, and data modeling.
- Experience working in Agile/Scrum teams and using CI/CD tools (Jenkins, GitLab, CodePipeline).
Skills
javapythonawsapache sparkhadoopkafkasqlagile