Audio · 2026-03-13 · 6 moments

Dylan Patel — Deep dive on the 3 big bottlenecks to scaling AI compute

Plus, why an H100 is worth more today than 3 years ago ✦ AI generated

timeline · colored by role

Anecdote

Anthropic's compute team, staffed by former Google employees, spotted a supply dislocation and locked in a huge TPU allocation before Google's own leadership grasped how much demand its AI business (and Anthropic's) would generate, forcing Google to scramble back to TSMC for more capacity once it woke up.

Dylan recounts an information-asymmetry story: Anthropic's ex-Google compute leads recognized a capacity opportunity and negotiated a massive TPU deal before Google itself foresaw its own revenue inflection, leading Google to belatedly rush back to TSMC for more wafer allocation.

transcript

Dylan Patel: The chain of events, at least from our data that we found, was in early Q3, over the course of six weeks, we saw capacity on TPUs go up by a significant amount. It went up multiple times in those six weeks. There were multiple requests. Google even had to go to TSMC and explain to them why they needed this increase in capacity because it was so sudden. A lot of that capacity increase was for selling to Anthropic.

Prediction◆

By 2028-2029 the single biggest constraint on scaling AI compute will be ASML's EUV lithography tools: ASML can only build about 70 this year, 80 next year, and just over 100 by 2030 even under very aggressive supply chain expansion.

Dylan explains that once mobile/PC chip capacity can no longer be shifted to AI, the ultimate ceiling on AI chip production becomes ASML's EUV tool output — capped near 100 machines a year by 2030 no matter how aggressively the industry expands.

transcript

Dylan Patel: To scale compute further, there are different bottlenecks this year and next year, but ultimately by 2028 or 2029, the bottleneck falls to the lowest rung on the supply chain, which is ASML. ASML makes the world's most complicated machine: an EUV tool. The selling price for those is $300-400 million. Currently, they can make about 70. Next year, they'll get to 80. Even under very aggressive supply chain expansion, they only get to a little bit over 100 by the end of the decade.

provides context · 1

Prediction

Whether the US or China ends up dominating AI compute hinges on takeoff speed: a fast AI takeoff lets the West's compounding revenue and compute investment widen its lead, but if AI progress is slow enough to stretch out toward 2035, China's more vertically indigenized semiconductor supply chain could let it eventually scale past the West.

Dylan argues fast AI progress favors the US because Anthropic-style revenue and compute investment compound faster than China can build comparable lab-scale infrastructure, but a slow enough timeline gives China's fully indigenized supply chain room to catch up and overtake the West's more fragmented, multi-country one.

transcript

Dylan Patel: If and when Anthropic 10Xs revenue again—and I think our answer would be when, not if—China doesn't have the compute to deploy at that scale. So there is some sense that we're in a fast takeoff. It's not like we're talking about a Dyson sphere by X date, it's more like the revenue is compounding at such a rate that it does affect economic growth.

Data

The AI-driven memory crunch is set to add roughly $100-250 to the retail cost of an iPhone, as DRAM per-gigabyte pricing roughly triples and NAND prices rise in tandem.

Dylan walks through the math showing how the AI memory crunch flows into consumer electronics: DRAM cost per gigabyte tripling turns a roughly $50 memory bill-of-materials into $150 for a 12GB iPhone, a cost increase Apple can only partly absorb.

transcript

Dylan Patel: I believe an iPhone has 12 gigabytes of memory. Each gig used to cost roughly $3-4, so that's $50. But now the price of memory has tripled. Let's say it's $12 per gig for DDR. Now you're talking about $150 versus $50. That's a $100 increase in cost for Apple.

Claim

Power will not be the binding constraint on US AI compute scaling, because unconventional generation (behind-the-meter turbines, reciprocating and ship engines, fuel cells, solar-plus-battery) can unlock roughly 20% of the terawatt-scale US grid that today sits idle except during rare peak-demand spikes.

Dylan argues the US can scale AI power well beyond current levels by tapping unconventional generation sources to absorb grid capacity normally reserved for rare peak demand, meaning power — unlike chips — isn't fundamentally supply-constrained.

transcript

Dylan Patel: Today, data centers are only 3-4% of the power of the US grid, and by 2028 they'll be 10%. But if you can unlock 20% of the US grid like this, it's not that crazy. The US grid is terawatt-level, not hundreds-of-gigawatts-level. So we can add a lot more energy.

Claim◆

An H100 GPU is worth more in economic value today than it was three years ago, because dramatically better and cheaper-to-serve models like GPT-5.4 extract far more usable value out of the same chip than GPT-4 ever could.

Dylan argues that because model quality and efficiency have improved so dramatically (GPT-5.4 vs GPT-4), the same H100 hardware now serves far more valuable workloads, inverting the usual assumption that GPUs depreciate in value over time.

transcript

Dylan Patel: When you look at an H100, it can serve more tokens per GPU of 5.4 than if you had ran GPT-4 on it. So it's producing more tokens of a model that is of higher quality. What is the maximum TAM for GPT-4 tokens? Maybe it was a few billion dollars, maybe it was tens of billions of dollars. Adoption takes time. For GPT-5.4, that number is probably north of a hundred billion.

Highlight slides

ASML: The Ultimate AI Compute Bottleneck✦ from: By 2028-2029 the single biggest constraint on scaling AI compute will be ASML's EUV lithography tools: ASML can only build about 70 this year, 80 next year, and just over 100 by 2030 even under very aggressive supply chain expansion.EUV Output Can't Keep Pace With AI Demand✦ from: By 2028-2029 the single biggest constraint on scaling AI compute will be ASML's EUV lithography tools: ASML can only build about 70 this year, 80 next year, and just over 100 by 2030 even under very aggressive supply chain expansion.H100s Get More Valuable, Not Less✦ from: An H100 GPU is worth more in economic value today than it was three years ago, because dramatically better and cheaper-to-serve models like GPT-5.4 extract far more usable value out of the same chip than GPT-4 ever could.TAM Inflection: GPT-4 vs GPT-5.4✦ from: An H100 GPU is worth more in economic value today than it was three years ago, because dramatically better and cheaper-to-serve models like GPT-5.4 extract far more usable value out of the same chip than GPT-4 ever could.

Related episodes

Article · Import AI

Import AI 463: Self-improving robots; a 10k Chinese GPU cluster; and an elegiac essay for the human era

→

Article · Import AI

Import AI 464: Fables writes GPU kernels; AI automation; and analog computation

→

Article · Import AI

Import AI 461: "Alignment is not on track"; FrontierCode; and synthetic research interns

→