ATRIUMsearch → argument graph
DataVideo · 46:52 — 48:22

AI coding tools have not made developers dramatically more productive; a study found only a tiny uptick in the actual amount of high-quality software being shipped.

Contrary to hype about massive productivity multipliers from AI coding, Howard cites his own study showing only a marginal increase in what people actually ship. ✦ AI generated

Jeremy Howard · Machine Learning Street Talk · 2026-03-03 · original ↗

starts at this moment · 46:52

No one's actually creating 50 times more high-quality software than they were before. So, we've actually just done a study of this and there's a tiny uptick, tiny uptick in what people are actually shipping. That's the facts.

verbatim transcript · starts at 46:52

Transcript · around this moment

46:32sure you have this in your company as well. Like you need to that this knowledge acquisition bottleneck is a real serious problem in in organizations. So when it's just me, I I I think I'm probably about 50 times more productive using claude code. It's absolutely magic and I can see why people are so excited about it. But people don't seem to understand the bottleneck and and how that doesn't

46:52really translate to many real world organization. >> No one's actually creating 50 times more highquality software than they were before. So, we've actually just done a study of this and there's a tiny uptick tiny uptick in what people are actually shipping. That's the facts. Obviously, I'm an enthusiast of of AI and what it can do, but also my wife Rachel recently pointed out in an article, all of the pieces that make gambling

47:26addictive are present in >> Yeah. dark flow. >> I was going to bring that up. You have to tell us about >> coding. >> Yeah, >> it's this really awkward situation where it's very almost everybody I know who got very enthusiastic about AI powered coding in recent months have totally changed their mind about it when they finally went back and looked at like how much stuff that I built during those

47:53days of great enthusiasm am I using today? Are my customers using today? am I making money from today? Almost all the money is being made by influencers, you know, or by the companies that produce the tokens. The thing about AI based coding is that it's like a slot machine in that you you have an illusion of control. You know, you can get to craft your prompt and your

48:22list of MCPs and your skills and whatever. And then but in the end, you pull the lever, right? You put in the prompt and something comes back and it's like cherry cherry. It's like oh next time I'll change my prompt a bit. I'll add a bit more context. Pull the lever again. Pull the lever again. It's the stochastic thing. You get the occasional win that's like, "Oh, I won. I got a

48:48feature." So, it's got it's got all these hallmarks of like loss disguised as a win. um somewhat stochcastic uh feeling of control, all the stuff that um gaming companies try to engineer into their gaming rooms. Now, none of that means that AI is not useful, but gosh, it's hard to tell. >> I know. And and Rachel, just just to be clear to she also said that one of the

Around this claim