SambaNova Strategic Collaboration
Supermicro and SambaNova announced a strategic collaboration to build and sell AI disaggregated inference systems. The partnership involves developing leading-edge SN50 inference racks that operate alongside NVIDIA GPUs as part of Supermicro's Data Center Building Block Solutions (DCBBS). The collaboration is initially planning for 10,000 SN50 RDU chips from SambaNova.
In these systems, the workload is divided between different hardware components. According to SambaNova, the NVIDIA GPUs handle the prefill phase, which is a compute-bound task that reads the prompt. The SambaNova SN50 Reconfigurable Dataflow Units (RDUs) handle the decode phase, which generates the response one token at a time and is memory-bound. SambaNova says that assigning each phase to suitable hardware delivers faster and more efficient inference than either hardware type alone.
"Our collaboration with SambaNova expands our broad DCBBS portfolio by enabling heterogeneous architectures that integrate GPU computing with specialized inference acceleration, helping customers deploy efficient, scalable AI solutions," said Charles Liang, president and CEO of Supermicro.