onsite
Software Engineer, Managed Service for Apache Spark
Software Engineer
As a Software Engineer, you will architect, implement, and operate the Dataproc Control Plane. You will build customer-facing features for running Hadoop, Spark, Flink, AI, and ML in the cloud, while ensuring global scale resource provisioning and cluster orchestration. This role also involves designing features for Dataproc and ensuring high availability, fault tolerance, and state management.
About the role
Architect implement and operate Dataproc Control Plane
Build customer facing features for running Hadoop Spark Flink AI ML in the cloud
Design and implement features for Dataproc
Enable global scale resource provisioning and cluster orchestration
Ensure high availability fault tolerance and state management
Skills
API IntegrationApache FlinkApache HadoopApache SparkApache YARNBigQueryCluster OrchestrationCompute EngineData GovernanceData LineageDistributed ComputingDistributed File SystemFault ToleranceFile SystemGoogle ComputeGoogle Compute EngineHadoop Distributed File SystemHigh AvailabilityPrestoState management