Senior Software Engineer - AI Inference

We are seeking a Senior Software Engineer – AI Inference to advance open-source LLM serving by contributing directly to upstream inference engines like vLLM and SGLang, ensuring they run best-in-class on NVIDIA GPUs and systems, and by improving the underlying stack that enables high-throughput, low-latency inference at scale.

Requirements

• Contribute features, fixes, and optimizations upstream to vLLM/SGLang
• Implement and optimize inference-runtime capabilities
• Profile and improve hot paths across layers
• Improve multi-GPU inference performance and reliability
• Build and maintain performance and correctness regression tests
• Collaborate with model, platform, and SRE teams
• Strong programming skills in Python plus C++ and/or CUDA
• Experience with LLM inference/serving stacks
• Experience with profiling and performance investigation
• Familiarity with distributed systems concepts and concurrency

Benefits

• Base salary determined based on location, experience, and similar positions
• Eligible for equity and benefits