LLM Pre-training & Distributed Engineer (AI Infrastructure)

Seeking a highly skilled LLM Pre-training & Distributed Systems Engineer with expertise in GPU clusters and system engineering to ensure efficient and reliable training processes.

Requirements

• Orchestrate distributed training runs across 1,000+ GPUs using PyTorch, DeepSpeed, or Megatron-LM.
• Optimize networking (InfiniBand/RDMA) and memory management to prevent out-of-memory errors.
• Automate checkpointing and failure recovery during month-long training runs.