RL's inefficiency is best understood as bits-per-FLOP, which decomposes into samples-per-FLOP (which falls as trajectories get longer) times bits-per-sample (which is far lower for RL's win/loss signal than for a supervised learning label).

Dwarkesh Patel lays out a framework showing RL is doubly inefficient compared to supervised learning: fewer samples per FLOP as horizons lengthen, and far fewer bits of signal extracted per sample since a binary win/loss carries much less information than a full label distribution. ✦ AI generated

Dwarkesh Patel · Dwarkesh Podcast · 2026-05-15 · original ↗

You can think of bits per FLOP as samples per FLOP times bits per sample. What I mentioned a second ago is that the samples per FLOP go down as RL becomes more long-horizon. But this kind of naive RL is also terrible from a bits-per-sample perspective, at least compared to supervised learning.

verbatim transcript · starts at 132:02