onsite
Java Spark Developer - Software Engineer II
Software Engineer
We are seeking a Software Engineer II with strong Java expertise to design, develop, and optimize high‑performance data pipelines on Apache Spark, leveraging CI/CD practices and deep knowledge of concurrency and distributed systems.
About the role
Key Responsibilities
- Design, implement, and maintain scalable data processing pipelines using Java and Apache Spark.
- Collaborate with cross‑functional Agile teams to translate business requirements into technical solutions.
- Develop and manage CI/CD pipelines to ensure reliable, automated deployment of Spark applications.
- Optimize performance and resource utilization by applying advanced concurrency techniques and distributed system principles.
- Conduct code reviews, troubleshoot production issues, and continuously improve code quality and reliability.
Requirements
- 3+ years of professional experience developing Java applications, with a focus on big‑data processing.
- Proven expertise with Apache Spark (core, SQL, Structured Streaming) in production environments.
- Hands‑on experience building and maintaining CI/CD pipelines using tools such as Jenkins, GitLab CI, or Azure DevOps.
- Strong understanding of concurrency, multithreading, and distributed system design patterns.
- Familiarity with Agile methodologies and collaborative development tools (Git, JIRA, Confluence).
Skills
javaapache sparkcicd