onsite
Software Engineering LMTS - Hyderabad Java Spark AI/ML
Software Engineer
Lead software engineer driving data‑centric solutions with Java, Spark, and AI/ML on AWS and Azure, leveraging Apache Iceberg for scalable data lake management.
About the role
Key Responsibilities
- Design, develop, and maintain high‑performance Java applications that process large datasets using Apache Spark.
- Implement and optimize data pipelines on AWS and Azure, ensuring reliability, scalability, and cost efficiency.
- Integrate AI/ML models into production workflows, collaborating with data scientists to translate research into scalable services.
- Utilize Apache Iceberg for schema evolution, partitioning, and ACID transactions in data lake environments.
- Participate in Agile ceremonies, code reviews, and continuous improvement initiatives.
Requirements
- 5+ years of professional software development experience, primarily in Java.
- Strong expertise in Apache Spark, including Spark SQL, DataFrames, and streaming.
- Hands‑on experience with AWS services (EMR, S3, Glue, SageMaker) and Azure equivalents.
- Proficiency with Apache Iceberg and modern data lake architectures.
- Solid understanding of AI/ML concepts and model deployment best practices.
Skills
javaapache sparkawsazure