Senior Backend Engineer, Inference Platform

Together AI is building the Inference Platform to power generative AI models. The role involves optimizing request routing, scaling systems, designing trade-offs between latency and throughput, and engineering features for efficient inference. The team works with LLMs, multimodal models, and image processing in a distributed environment, collaborating with research and the open-source community.

Requirements

• 5+ years of demonstrated experience building large-scale, fault-tolerant, distributed systems and API microservices.
• Strong background in designing, analyzing, and improving efficiency, scalability, and stability of complex systems.
• Excellent understanding of low-level OS concepts (multi-threading, memory management, networking, and storage performance).
• Expert-level programming in one or more of: Rust, Go, Python, or TypeScript.
• Experience working with the open source ecosystem around inference is highly valuable.

Benefits

• Competitive compensation
• Equity
• Health insurance