FactAudio · 105:47 — 107:17
Being first to achieve a result always costs vastly more compute than reproducing it later, because followers can lean on tricks like distillation and other crutches the original team didn't have.
Eric Jang notes he rebuilt a strong Go bot for about $10K in rented compute — versus AlphaGo Zero's roughly 3E23 FLOPS — because being first to solve a problem is inherently far more expensive than catching up once someone else has already solved it. ✦ AI generated
Eric Jang · Dwarkesh Podcast · 2026-05-15 · original ↗
plays this moment only · 105:47 — 107:17
Elicited by
“Is your sense that they did a bad job training it, if you can do it in $10K now?”
The compute required to be the first to do something is always much larger than the compute it takes to catch up. It’s the same story playing out in LLMs. Once someone else has done it, you can use tricks like distillation. You can use all sorts of crutches to bootstrap your way to success.
verbatim transcript · starts at 105:47
Transcript · around this moment
105:47– Why doesn’t MCTS work for LLMs
105:47– Why doesn’t MCTS work for LLMs
Around this claim