Software Engineer
Salad Technologies · Remote · Full-time SaladCloud runs inference on thousands of geodistributed workstation and consumer GPUs — NVIDIA and AMD — that nobody else can use.
Staff Software Engineer, AI Inference Gateway
Salad Technologies · Remote · Full-time
The role
SaladCloud runs inference on thousands of geodistributed workstation and consumer GPUs — NVIDIA and AMD — that nobody else can use. Sitting in front of that fleet is a unique AI Gateway written in Rust, built on Pingora and WireGuard, that serves real-time and batch inference and embeddings to customers as a single, reliable API. You'll own it.
This is an infrastructure role with real autonomy. You'll design and ship the systems that decide how requests are fulfilled, when to add or drop capacity, and how to bill for every token — and you'll be the person who knows whether a new model, quantization, or GPU is actually worth running.
What you'll do
The team
You'll join a small team — five engineers plus the CTO, who still writes code, and two deeply technical product managers who ship their own fixes rather than queue them for you. You'll report directly to the CTO. Everyone on the engineering team has 10+ years of experience and has built systems that already run at massive scale — with minimal incidents in a typical year, because we ship things that are production-worthy the first time.
You'll be the first hire with inference-gateway expertise and you'll be asked to own the whole surface. You won't do it alone: your teammates are eager to learn the domain, will pair on design and integrations, and share after-hours support. The bar is high; so is the support.
We've listed a lot below. If you're strong in two of the first three — Rust, LLM inference servers, distributed systems — apply, and expect to learn the rest fast with people who'll help.
What we're looking for
Posted September 21, 2026