onsite
Software Engineer, Managed Service for Apache Spark
Software Engineer
This role involves architecting, implementing, and operating the Dataproc Control Plane. You will build customer-facing features for running Hadoop, Spark, Flink, AI, and ML in the cloud. The position also focuses on designing and implementing features for Dataproc, enabling global scale resource provisioning and cluster orchestration, and ensuring high availability, fault tolerance, and state management.
About the role
Architect implement and operate Dataproc Control Plane
Build customer facing features for running Hadoop Spark Flink AI ML in the cloud
Design and implement features for Dataproc
Enable global scale resource provisioning and cluster orchestration
Ensure high availability fault tolerance and state management
Skills
API IntegrationApache FlinkApache HadoopApache SparkApache YARNBigQueryCluster OrchestrationCompute EngineData GovernanceData LineageDistributed ComputingDistributed File SystemFault ToleranceFile SystemGoogle ComputeGoogle Compute EngineHadoop Distributed File SystemHigh AvailabilityPrestoState management