The concept of a 'maze' isn't a native primitive present in a newborn's mind — it's an abstraction humans have built up over generations of problem-solving, and injecting such accumulated abstractions into an LLM lets it skip many levels of abstraction that would otherwise have to be learned from scratch.
Michael distinguishes innate core-knowledge priors from generationally-accumulated concepts like 'maze,' arguing that giving LLMs access to these accumulated human abstractions lets them bypass levels of abstraction-building. ✦ AI generated
Michael · Machine Learning Street Talk · 2026-07-01 · original ↗
starts at this moment · 30:01
In the example that we saw we have a maze, right? There's no native primitive in a in a newborn that says this is a maze. But this has been over generations of problem solving through humans uh very much established concept. And if we can inject this into an LLM, we can skip many levels of abstraction.
verbatim transcript · starts at 30:01
30:01that we accumulate over generations through education and so on. And those abstractions are way more powerful. In the example that we saw we have a maze, right? There's no native primitive in a in a newborn that says this is a maze. But this has been over generations of problem solving through humans uh very much established concept. And if we can inject this into an LLM, we can skip many levels of abstraction.
30:31Um, and I would say that's why when we play these games, some of them for the first time, they're also very difficult for humans because if this if this abstract, if this is done well and there's nothing like a maze in a game and it really c boils down to the core knowledge prior, then there's it's very difficult even for a grown-up adult to to synthesitize the correct rules.
30:53However, most often um and that may be a failure of of uh Arc a little bit in its design is that these games usually have some sort of oh, you're shooting a ball or you're solving a maze, you're you have an enemy, right? And this these are like as as pure as they're trying to be, they're never going to be pure because they're made by humans. Um, and that's
31:14why we can do so well with LLMs, I think, because a lot of these like the maze is the canonical example or uh uh I think like those can be really well done with LLM. They have these fractured kind of, you know, um fractionated understandings. But isn't it interesting that when humans use LLMs, we can make them act as if they understand because we can say, well, um you can put a
31:37framing in in the prompt. You can say, think about this problem like it's a maze or think about this problem like it's um tic-tac-toe or something like that. And what you're doing in the prompt is you're basically setting constraints on its generation. >> Yeah. And then you know like the more you work on a software engineering project you know like it doesn't quite understand. You put more constraints in
31:53you update the prompt and it's like it converges and after a while you don't need to repeat yourself. It it it understands and so it's almost as if when you get LLMs in the mood when you make them track this perspective where it respects the constraints then it understands. But like the million-dollar question is how can we just make it understand autonomously? How can we give it a novel domain and do what we do