Staff Software Engineer, AI Inference Gateway

Salad Technologies is seeking a Staff Software Engineer to own and scale their AI Inference Gateway, a unique system built in Rust that manages inference across a geodistributed fleet of consumer and workstation GPUs. The role involves designing request routing, autoscaling algorithms, and observability systems while benchmarking new models and hardware to optimize performance and cost. This infrastructure position offers significant autonomy to shape the platform that serves real-time and batch inference to customers.