Video · 2026-03-03 · 1h 27m · 6 moments

The Dangerous Illusion of AI Coding? - Jeremy Howard

✦ AI generated

timeline · colored by role

Example

What looks like novel AI-built engineering, such as a 'clean room' Rust C compiler, is actually interpolation between existing training data rather than genuine creativity or extrapolation.

Using Anthropic's Claude-built C compiler as an example, Howard argues the model isn't creating something new — it's finding a 'nonlinear average point' between existing LLVM code and Rust idioms, a style-transfer problem, not creativity.

transcript

Jeremy Howard: That misunderstands how LLMs work, right? which is all of Chris's work was in the training data many many times LLVM is used widely and lots and lots of things are built on it um including lots of C and C++ compilers converting it converting it to Rust is an interpolation between parts of the training data you know it's a style transfer problem.

rebuts · 1

Definition

Coding (producing code text) and software engineering are fundamentally different disciplines, so AI fluency at generating code doesn't mean AI is good at software engineering.

Howard argues executives like Dario Amodei and Elon Musk conflate typing code with the much broader discipline of software engineering when predicting AI will replace engineers.

transcript

Jeremy Howard: Software engineering is a unusual discipline and a lot of people mistake it for being the same as typing code into an IDE. Coding is another one of these style transfer problems. You take a specification of the problem to solve and you can use your compositional creativity to find the parts of the training data which interpolated between them solve that problem and interpolate that with syntax of the target language and you get code.

extends · 1

Claim

Large language models don't actually understand things; they convincingly imitate understanding, and this pretense works fine until it suddenly and completely breaks down.

Howard resolves the online debate over whether LLMs 'understand' by saying both sides are right: models cosplay understanding convincingly, which is functionally fine until the pretense collapses.

transcript

Jeremy Howard: There's often this these arguments online between people who are like, "LMs don't understand anything. They're just pretending to understand." And then other people are like, "Don't be ridiculous. Look what this LLM just did for me." Right? And the funny thing is they're both right. LLM's cosplay understanding things. They pretend to understand things.

explains mechanism · 1rebuts · 6supports · 1

Data◆

AI coding tools have not made developers dramatically more productive; a study found only a tiny uptick in the actual amount of high-quality software being shipped.

Contrary to hype about massive productivity multipliers from AI coding, Howard cites his own study showing only a marginal increase in what people actually ship.

transcript

Jeremy Howard: No one's actually creating 50 times more high-quality software than they were before. So, we've actually just done a study of this and there's a tiny uptick, tiny uptick in what people are actually shipping. That's the facts.

rebuts · 2

Claim◆

AI-assisted coding gives programmers an illusion of control while the actual code produced is often not truly understood by anyone, much like pulling a slot machine lever.

Jeremy Howard compares AI coding tools to gambling slot machines: crafting prompts, MCPs, and skills feels like skill and control, but pulling the trigger produces a stochastic result and code nobody fully understands.

transcript

Jeremy Howard: The thing about AI based coding is that it's like a slot machine in that you have an illusion of control. You know, you can get to craft your prompt and your list of MCPs and your skills and whatever. And then but in the end, you pull the lever, right? You put in the prompt and something comes back.

explains mechanism · 1rebuts · 2supports · 1

Claim

The real danger of powerful AI is not autonomous takeover but the centralization of that power in a few companies or governments, since power-hungry actors will inevitably seek to monopolize and abuse it.

Rebutting the 'autonomous AI takeover' existential-risk narrative associated with figures like Eliezer Yudkowsky, Howard argues the true threat is concentrating powerful AI in a few hands, which power-hungry people will inevitably seize and exploit.

transcript

Jeremy Howard: Even if it turns out to be incredibly powerful, right? Like I don't I don't even want to argue about whether it's going to be powerful because that's speculative. Even if it's going to be incredibly powerful, you still shouldn't centralize all of that power in the hands of one company or the government.

supports · 1

Highlight slides

AI Coding Tools: Hype vs. Reality✦ from: AI coding tools have not made developers dramatically more productive; a study found only a tiny uptick in the actual amount of high-quality software being shipped.Study: Marginal Impact Only✦ from: AI coding tools have not made developers dramatically more productive; a study found only a tiny uptick in the actual amount of high-quality software being shipped.AI Coding as Slot Machine✦ from: AI-assisted coding gives programmers an illusion of control while the actual code produced is often not truly understood by anyone, much like pulling a slot machine lever.Illusion vs. Reality✦ from: AI-assisted coding gives programmers an illusion of control while the actual code produced is often not truly understood by anyone, much like pulling a slot machine lever.

Related episodes

Video · a16z Podcast

What looks like novel AI-built engineering, such as a 'clean room' Rust C compiler, is actually interpolation between existing training data rather than genuine creativity or extrapolation.

Coding (producing code text) and software engineering are fundamentally different disciplines, so AI fluency at generating code doesn't mean AI is good at software engineering.

Large language models don't actually understand things; they convincingly imitate understanding, and this pretense works fine until it suddenly and completely breaks down.

AI coding tools have not made developers dramatically more productive; a study found only a tiny uptick in the actual amount of high-quality software being shipped.

AI-assisted coding gives programmers an illusion of control while the actual code produced is often not truly understood by anyone, much like pulling a slot machine lever.

The real danger of powerful AI is not autonomous takeover but the centralization of that power in a few companies or governments, since power-hungry actors will inevitably seek to monopolize and abuse it.

Ben Horowitz on the Global Race for Tech, Power, and Influence

6 months to live for open models

Kimi K3: The open-weights escalation