Claim◆Audio · 60:33 — 62:03

A ten-layer neural network can amortize and approximate an almost intractable search problem to very high fidelity — a breakthrough most people still don't fully appreciate, and one that hints our theoretical notions of computational hardness may be incomplete in practice.

Eric Jang argues AlphaGo's core breakthrough — compressing an intractable Go search into a 10-layer forward pass — is more profound than commonly recognized, and that the same phenomenon underlies AlphaFold and may complicate our understanding of NP-hardness. ✦ AI generated

Eric Jang · Dwarkesh Podcast · 2026-05-15 · original ↗

Share card ↓Export .md Export .txt

plays this moment only · 60:33 — 62:03

Elicited by

“I don’t know if you share that intuition where the more you understand it, the less impressive the accomplishment in 2017 seems.”

10 steps of neural network parallelized distributed-representation thinking is able to amortize and approximate to very high fidelity a nearly intractable search problem. This was a breakthrough that I think most people don’t even fully comprehend today, how profound that accomplishment is.

verbatim transcript · starts at 60:33

Transcript · around this moment

60:33– Self-play

✳ AlphaFold ✳ AlphaGo

Around this claim

Mechanism · 1

MCTS-based training works because it never asks the network to directly chase a win/loss signal — instead, for every single action taken, search produces a strictly better relabeled target, giving one clean supervision signal per action instead of a noisy win-rate credit-assignment problem.Eric Jang · Dwarkesh Podcast · conf 60%

Context · 1

Deep networks solve the curse of dimensionality by discovering hierarchical abstract variables in data, allowing them to generalize with polynomial rather than exponential amounts of data.Matthieu Wyart · Machine Learning Street Talk · conf 70%