A paper claims reinforcement learning for reasoning only changes 1-3% of tokens at high-entropy decision points, and the promoted tokens are always within the base model's top-5 alternatives.

Research suggests RL for LLM reasoning acts as a sparse reranker over existing base-model alternatives, with the proposed ReasonMaxxer method replicating gains at 1000x less compute. ✦ AI generated

AINews · Latent Space · 2026-08-17 · original ↗

A paper by Akgül (2026), ReasonMaxxer, claims RL-based reasoning improvements in LLMs mostly come from sparse policy corrections rather than newly learned reasoning: token-level analyses across model families/RL algorithms reportedly find only ~1–3% of token positions change, concentrated at high-entropy 'decision points.' It further claims the RL-promoted token is always already within the base model's top-5 alternatives, and proposes ReasonMaxxer, an RL-free contrastive/entropy-gated method using a few hundred base-model rollouts that allegedly matches or exceeds full RL on math benchmarks at roughly 1000x lower compute.

Read full article ↗excerpt · fair-use quotation