GPUs waste most of their memory bandwidth on inference because they synchronize data movement between kernels in software, whereas SambaNova's reconfigurable dataflow chips push memory-bandwidth utilization to 70-80% versus roughly 10-20% for GPUs.

100:47improvements peak improvements on HPM bandwidth and uh chip-to-chip communication by using the latest technology but then the key is how effectively do you use that bandwidth how effectively do you use that communication and do you make sure that you don't waste it right and so uh you know whereas uh GPU views are often running at you know maybe 10 to 20% of the uh capabilities of the resources

101:20right uh the bandwidth and and and and the uh the memory bandwidth and and and the communication resources our goal in a Samanova system is to push that to be 70 to 80% of of the peak and so the idea is yeah everybody Everybody wants more capabilities from the underlying uh resources, but the key is keeping that those resources as effectively used as possible. And of course that gives you

101:50more benefit for for the uh uh the cost that you spend on on on on providing uh higher memory bandwidth with the latest HPM and and uh higher signaling uh frequencies and and communication bandwidths between the chips using the latest uh uh variety of of and vlink and so >> for a lot of the chip users they uh they often focus on this number the model flops

102:21utilization the MFU. >> Um what what is your northstar as you guys are designing the chip like is that is that is that the strongest consideration or are some other metrics or numbers things that you focus on? No, the speed of light of of doing inference is really uh especially if you want high speed inference, right? So, everybody, you know, knows that we're in the aentic envir era and and so what one wants is