Inference Software Engineer - Collectives
Etched(3 months ago)
About this role
The Inference Software Engineer at Etched focuses on optimizing and formalizing collective communication methods (e.g., Send/Receive, AllReduce, Broadcast) within the company's innovative inference system. This role involves collaborating with systems and research teams to implement Mixture of Experts architectures, enhance Sohu's runtime for multi-node inference, and develop performance profiling tools. The ideal candidate should have strong expertise in Rust or C++, distributed systems, and high-performance networking, along with experience in designing and optimizing communication protocols.
Required Skills
- Collectives Optimization
- Cross-Functional Collaboration
- Model Implementation
- Expert Routing
- Communication Layers
- Runtime Scaling
- Performance Profiling
- Debugging Tools
- Bottleneck Identification
- Strong Proficiency in Rust
+18 more
About Etched
www.etched.comEtched is at the forefront of advanced computing technology with its groundbreaking product, Sohu, the world's first transformer ASIC. By etching transformer architecture directly into silicon, Etched delivers server solutions that provide dramatically faster and more cost-effective AI model inference compared to traditional GPU-based systems. Their innovative technology is designed to optimize the performance of AI applications, enabling unprecedented processing capabilities for next-generation models. With a commitment to pushing the boundaries of what's possible in AI, Etched is poised to revolutionize the industry.