Member of Technical Staff - Inference
Prime IntellectSan Francisco, California, United States
Posted September 16, 2025
Full Time
About the Company
Prime Intellect democratizes AI development by providing an accessible platform to find global compute resources and train state-of-the-art models through distributed training. The company enables collective ownership of open AI innovations, including language models and scientific breakthroughs, fostering collaboration and innovation in the field.
Job Description
Prime Intellect is building the open superintelligence stack, aggregating and orchestrating compute resources to enable researchers, startups, and enterprises to create, train, and deploy RL models at scale. They are seeking a Member of Technical Staff to focus on building and optimizing their Infrastructure to serve LLMs at scale.
Requirements
- 3+ years building and running large-scale ML/LLM services
- Hands-on experience with vLLM, SGLang, TensorRTāLLM
- Familiarity with distributed serving infrastructure like NVIDIA Dynamo
- Understanding of prefill vs. decode, KVācache behavior, batching, sampling, and speculation
- Comfort with debugging CUDA/NCCL, drivers, containers, service mesh/networking, and storage
- Full-Stack Debugging
- Infrastructure Skills (Python, PyTorch, Cloud/Automation, Kubernetes)
- Infrastructure Skills (Rust, C++, Kafka/PubSub, Redis/gRPC/Protobuf, Prometheus/Grafana)
- Infra & Config Automation (Terraform/Ansible, Infrastructure-as-Code)
Benefits
- Competitive compensation
- Flexible work arrangement
- Professional development budget
- Regular team off-sites and conference attendance
