Senior Software Engineer- Big Data Compute Platform

About the Role

The Compute Platform team at LinkedIn builds and manages the platform that trains ML models and runs scheduled data processing, which serves insights and member experiences. We run one of the largest compute clusters on the planet spanning tens of thousands of nodes, processing petabytes of data in millions of containers, daily. We extensively collaborate with open source and influence open-source communities through a team of contributors, committers, and committee members in Apache, CNCF, Linux Foundation, and other open-source projects.

We are looking for talented engineers to build the next generation of our compute platform in Cloud. Your work will impact thousands of users inside and outside the company and give you an opportunity to work with industry leaders in this field. If you are passionate about building and designing scalable, highly available, and mission-critical distributed systems, then this is the place for you.

Responsibilities

• Build and operate the compute platform for LinkedIn, that runs in the cloud.
• Design and implement large-scale distributed systems keeping security and compliance in mind.
• Optimize the performance of distributed systems at scale.
• Improve the observability and understandability of these systems.
• Work closely with the open-source community (e.g Apache Hadoop, Kubernetes)
• Collaborate with other teams to incorporate their use cases.
• Mentor and lead a group of junior engineers
• Identify, leverage, and successfully evangelize opportunities to improve engineering productivity

Basic Qualifications

• BS in Computer Science or related technical field or equivalent technical experience
• 6+ years of industry experience
• 4+ years of experience leading projects and building large scale distributed systems

Preferred Qualifications

• 4+ years of relevant work experience
• MS or PhD in Computer Science or related technical discipline
• Experience working with open-source projects and/or academic research in large-scale distributed systems.
• Published work in academic conferences or industry circles.
• Experience in cloud computing (e.g. Azure).
• Experience with Hadoop ecosystem (YARN, HDFS, Hive, Spark, Presto) or similar big data ecosystems
• Experience with Kubernetes and Docker.
• Expert knowledge in computer science, with strong competencies in data structures, algorithms, and software design.