remote
Senior Big Data Software Engineer - Optum
Software Engineer
Senior engineer designing, building, and optimizing large‑scale data pipelines and platforms using Java/Scala, Spark, Hadoop, and cloud services to enable health data integration and analytics.
About the role
Key Responsibilities
- Design and develop high‑performance, scalable data pipelines and processing frameworks using Java, Scala, and Apache Spark.
- Implement and maintain big‑data storage solutions on Hadoop ecosystems and cloud platforms (AWS S3, EMR, Redshift).
- Integrate real‑time data streams with Apache Kafka and other messaging systems to support analytics and reporting.
- Collaborate with data scientists, product owners, and cross‑functional teams to translate business requirements into robust data solutions.
- Ensure code quality, reliability, and performance through automated testing, code reviews, and continuous integration/continuous deployment (CI/CD) practices.
Requirements
- 5+ years of professional experience developing big‑data applications with Java and/or Scala.
- Strong expertise in Apache Spark, Hadoop, and related ecosystem tools (Hive, HBase, Parquet).
- Hands‑on experience with cloud services, preferably AWS (S3, EMR, Lambda, CloudFormation).
- Proficiency in building and managing data pipelines using Kafka or similar streaming platforms.
- Solid understanding of relational and NoSQL databases, SQL query optimization, and data modeling.
Skills
javascalaapache sparkhadoopawskafka