What looks like novel AI-built engineering, such as a 'clean room' Rust C compiler, is actually interpolation between existing training data rather than genuine creativity or extrapolation.
Using Anthropic's Claude-built C compiler as an example, Howard argues the model isn't creating something new — it's finding a 'nonlinear average point' between existing LLVM code and Rust idioms, a style-transfer problem, not creativity. ✦ AI generated
Jeremy Howard · Machine Learning Street Talk · 2026-03-03 · original ↗
starts at this moment · 24:07
That misunderstands how LLMs work, right? which is all of Chris's work was in the training data many many times LLVM is used widely and lots and lots of things are built on it um including lots of C and C++ compilers converting it converting it to Rust is an interpolation between parts of the training data you know it's a style transfer problem.
verbatim transcript · starts at 24:07
24:07Chris's work was in the training data many many times LLVM is used widely and lots and lots of things are built on it um including lots of C and C++ compilers converting it converting it to Rust is an interpolation between parts of the training data you know it's a style transfer problem um so it's definitely compositional creativity at most if you can call it creative at all
24:34and you actually see it when you look at the the repo that it created, it's copied parts of the LL VM code which today Chris says like, "Oh, I made a mistake. I shouldn't have done it that way. Nobody else does it that way." You know, oh wow, look, they're the only other one that did it that way. That doesn't happen accidentally. That happens because you're not actually being
24:59creative. you're actually just finding the kind of nonlinear average point in your training data between like Rust things and building compiler things. >> All of that is true. I mean first of all I think we shouldn't underestimate the size of how big this combinatorial creativity is. So all of that is true. So the code is on the internet but also they had a whole bunch of tests which
25:21were scaffolded which meant that every single time some code was was committed they could run the test and they and they basically had a critic and they could then do this autonomous feedback loop. So in in a sense it's very similar to the recent research by open AAI and and Gemini where you you're you're trying to solve a problem in math and you already have an evaluation function.
25:42The same on the AR prize, right? You have an evaluation function and what people discount is even knowledge of what the evaluation function is is partial knowledge of the problem. So you can then brute force search. You can use the statistical pattern matching, use the verifier as a constraint and you can actually >> and they don't even need to do that, right? like they you literally already
26:01know how to pass those tests because there's lots of software that already does it. >> So, it just uses that and translates them to Rust. Like that's that's all it did. Um which is impressive. >> Yeah. >> Um and if you I'm much less familiar with math than I am computer science, but from talking to mathematicians, they tell me that that's also what's happening with like Erdos problems and