A base model's accuracy on a math benchmark can jump from 15% to 50% after only 50 steps of RLVR training, showing that RL is unlocking latent pre-trained knowledge rather than teaching genuinely new mathematical understanding.
Sebastian recounts training Qwen 3's base model with RLVR on MATH-500 and watching accuracy leap from 15% to 50% in minutes, arguing this proves RL mainly unlocks pre-existing knowledge rather than instilling new understanding. ✦ AI generated
Sebastian Raschka · Lex Fridman · 2026-01-31 · original ↗
starts at this moment · 104:32
I was training the Qwen 3 base model with RLVR on MATH-500. The base model had an accuracy of about 15%. Just 50 steps, like in a few minutes with RLVR, the model went from 15% to 50% accuracy.
verbatim transcript · starts at 104:32
104:3215% to 50% accuracy. And the model... You can't tell me it's learning anything fundamentally about math in - The Qwen example is weird because there've been two papers this year, one of which I was on, about data contamination in Qwen and specifically that they train on a lot of this special mid-training phase that we should take a minute on, because it's weird because they train on problems that are almost identical to MATH.
104:53- Exactly. And so you can see that basically the RL, it's not teaching the model any new knowledge about math. You can't do that in 50 steps. So the knowledge is already there, in the pre-training, you're just unlocking it. - I still disagree with the premise because there's a lot of weird complexities that you can't prove because one of the things that points to weirdness is that if you take the Qwen 3 so-called base
105:14model and you... You could Google like "math dataset, Hugging Face", and you could take a problem and what you do if you put it into Qwen 3 base... All these math problems have words, so it'd be like "Alice has five apples and takes one... and gives three to whoever," and there are these word problems. With these Qwen-based models, why people are suspicious of them is if you change the numbers but keep the words-
105:36Qwen will produce, without tools, will produce a very high accuracy decimal representation of the answer, which means there's some... At some time, it was shown problems that were almost identical to the test set, and it was using tools to get a very high precision answer, but a language model without tools will never actually have this. So it's kind of been this big debate in the research community: how much of these reinforcement learning
106:03papers that are training on Qwen and measuring specifically on this math benchmark, where there's been multiple papers talking about contamination, is like, how much can you believe them? And I think this is what caused the reputation of RLVR being about formatting, because you can get these gains so quickly, therefore it must already be in the model. But there's a lot of complexity here that we... It's not really like controlled experimentation, so we don't really know.
106:26- But if it weren't true, I would say distillation wouldn't work, right? I mean, distillation can work to some extent, but the thing is that is, I think, the biggest problem, and I research this contamination because we don't know what's in the data. Unless you have a new dataset, it is really impossible. And the same, you mentioned the math dataset, where you have a question and then answer and an explanation is given, but then also even something simpler