For high-speed agentic AI, the real bottleneck isn't compute (floating-point matrix multiplication) but memory bandwidth for moving model weights and KV cache, and dataflow chips are designed to maximize utilization of that bandwidth to deliver fast, expensive 'premium tokens.'

102:50what we call premium tokens, right? So, premium tokens are tokens that you can charge the most money for because they are premium. And why are they premium? because they are they come from from very large models. So they're accurate, but they also are provided to you at high speed so that that they're they're useful from the point of view of of an agentic environment that needs many

103:14turns through the models and and potentially has vulnerable models that that interact, right? And so the question is, you know, uh how do you provide those premium tokens? And you provide those premium tokens by having very fast inference. And that is not going to be limited by floating point. It's going to be limited by moving the KV cache and the parameters from the uh HBM memory to the the the compute chip,

103:42right? And so we like to think of this as memory bandwidth utilization, right? So the speed of light is is one, right? is that you you use your memory bandwidth completely uh for one thing >> or maybe two things. Moving KB cache, moving uh uh the the the parameters from the the uh HPM to the the uh the compute uh unit uh so that you generate when you

104:13generate each token, right? And so in fact in many instances you're underus underutilizing the flops on the uh on the uh the GPU or the compute units because in fact you are running a memory memory bound problem right and in fact most a lot of of of the the problems uh that one runs on GPUs and especially inference are in fact not computebound. They are memory bound

104:44or memory and communication bound. >> Could we zoom out and just ask you to kind of taxonomize the whole chip space if you would? I mean this is a big question but I think people are familiar with things like Cerebras which obviously has this like giant chip and has like a ton of memory on chip. We've seen a number of instances where people are, you know, burning the transformer