Inception's newly launched Mercury 2 diffusion language model matches the quality of the best speed-optimized frontier-lab models (Haiku, Flash, Mini-class models) while being roughly 5-10x faster.

14:03results are extremely good like the latest model that we announced this week Mercury 2 is actually matching in quality some of the best speed optimized models from frontier labs so we'll think about the Haiku models the flash models mini models from OpenAI. So it's the at that quality level but again it's about 5 to 10x faster in terms of like the time it takes you to get an answer

14:29using a diffusion model versus an autoregressive model. Are are you able to to give us um an overview or a summary of like some of the mathematics that had that kind of make this work? In in at an intuitive level it's all somewhat similar in the sense that there is still a neural network that is trained to remove noise. Um it's just like the the noise process

14:51is no longer kind of like adding small numbers to the pixel intensities it's more like there is different kinds of noise processes that you can use one that works pretty well is basically one where you mask out tokens so you you kind of like uh hide them. You you take a sentence and then you remove some of the tokens you hide them from the neural network and then you ask

15:15the neural network can you predict what those tokens were? And so it's similar in some sense to next token prediction except that things are done out of order and the network needs to be able to use information from you needs to use context to the left and to the right and combine it in some interesting ways to figure out how to predict all these missing tokens from the from the

15:39sentence. So in some ways you're you're changing the definition of noise to one that makes sense in the context of text. Exactly exactly. And so in actually that kind of training objective is very similar to the BERT style models from from again many years ago but that was the thing that for a while you know was sort of like widely used in natural language processing people were

16:03training these neural networks exactly on the same objective this idea of oh let's train the network to predict some of the missing tokens you know if it in order to do that it really needs to understand the meaning of the other tokens and that's a good way to get representations. In that ICML paper that I mentioned basically we showed that well once you can do that you can also generate