Inference Stack Engineer

Inference Stack Engineer

(AI Systems / Compiler & Runtime)

We are building a next-generation AI inference stack designed for high-performance execution on modern and custom compute architectures. Our mission is to deliver industry-leading low-latency and high-throughput AI systems by designing and optimizing the full execution path — from model representation to hardware-level execution.

This is a deeply technical role at the intersection of compiler systems, AI runtimes, and high-performance computing .

You will work on core infrastructure that defines how modern AI models are executed efficiently at scale.

What you will do

Design and build components of an AI inference stack , from high-level model representation to low-level execution

Develop and extend a Python-based DSL for expressing AI workloads and kernels

Work on compiler infrastructure including:

IR design and transformation pipelines

graph lowering and optimization passes

backend code generation for target execution environments

Optimize model execution for:

latency

throughput

memory efficiency

numerical stability

Contribute to runtime systems responsible for model execution and scheduling

Profile and analyze inference workloads to identify system bottlenecks

Collaborate closely with hardware and systems engineers on execution efficiency

Influence architecture decisions for next-generation AI execution platforms

What we are looking for

Strong software engineering background (C++ and Python)

Experience with performance-critical systems or compiler-related work

Understanding of AI model execution (especially transformers / LLMs)

Familiarity with compute graphs, tensor operations, or execution frameworks

Ability to analyze complex systems end-to-end (model → runtime → hardware)

Experience working with large codebases and system-level debugging

Strong communication skills and ability to work in cross-functional teams

Nice to have

Experience with compiler frameworks such as:

LLVM

MLIR

Triton

TVM

XLA

Experience contributing to deep learning frameworks (PyTorch, TensorFlow, JAX)

Understanding of GPU or accelerator execution models

Experience with kernel optimization or operator-level performance tuning

Knowledge of distributed inference systems (e.g. NCCL, RPC-based serving)

Familiarity with hardware-aware optimizations (memory hierarchy, vectorization, scheduling)

What we offer

Work on the core execution layer of modern AI systems

Direct impact on inference performance of large-scale AI workloads

Collaboration with experts in compilers, systems, and AI infrastructure

Highly technical environment with strong engineering auto