Software Engineer, Networking - Inference

A Software Engineer in Networking - Inference at OpenAI is responsible for designing and building a high-performance load balancer for research inference that routes large AI models with precision and reliability. This role involves architecting the system for long-lived connections, developing traffic routing strategies, and implementing observability tools for complex distributed systems. The engineer will collaborate with researchers and ML engineers to ensure infrastructure decisions enhance model performance and will oversee the entire system lifecycle from design to deployment. Candidates should have extensive experience with distributed systems, particularly load balancers, and programming in systems languages like Rust or C++.