AI progress is less bottlenecked by expert human data than by algorithmic and infrastructure advances, so scaling up human expert data labeling wouldn't change progress much.
Ryan disputes that scaling up human expert data has driven AI progress, arguing the limiting factor on RL environments is knowing what to build and using AI labor, not more human expert labeling; Dwarkesh counters with market signals like the $2B Mechanize deal. ✦ AI generated
Ryan Greenblatt · Dwarkesh Podcast · 2026-08-11 · original ↗
plays this moment only · 16:52 — 34:02
“How are the AIs able to replicate the effect that expert human judgment currently seems to be playing in AI progress?”
My sense is that scaling up the amount of effort spent on getting expert human data has not been hugely important for AI R&D in general. In particular, over the last few years, we've been scaling up compute, scaling up people working at AI companies, and scaling up the amount of effort spent on data labeling. My sense is that if you removed the last two doublings or whatever of data generation from expert humans, that would not make a huge difference. A lot of what's been going on is people have been developing better ways to leverage humans and AIs to construct RL environments and going somewhere from that.
verbatim transcript · starts at 16:52
16:52– Is AI progress bottlenecked by human expert data?
34:02– Flat token prices suggest scaling has been slow
16:52– Is AI progress bottlenecked by human expert data?
34:02– Flat token prices suggest scaling has been slow