Claim◆Article

Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models, and a weaker sibling model can be jailbroken to recover the stronger model's hidden reasoning in plaintext.

The paper's central attack: frontier-model reasoning traces are returned to clients as encrypted blocks, then replayed into a weaker sibling model that—once jailbroken—reveals the stronger model's hidden chain of thought in plaintext. ✦ AI generated

Paper authors (via Hacker News, stolen-thoughts.com) · Simon Willison's Weblog · 2026-08-11 · original ↗

Share card ↓Export .md Export .txt

Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model's hidden reasoning in plaintext

Read full article ↗excerpt · fair-use quotation

▦ Anthropic ▦ Google ▦ OpenAI

Around this claim

Mechanism · 1

Every model under the same family shares the same encryption key, so blocks captured from one model can be fed back into the weakest family members, which can be jailbroken into outputting the unencrypted raw reasoning blocks.Paper authors (via Hacker News, stolen-thoughts.com) · Simon Willison's Weblog · conf 82%

This moment responds to

gives example → Claude Haiku 4.5 was the easiest model to attack, using a simple prompt instructing it to transcribe the reasoning verbatim inside a thinking-copy tag, helped by an assistant-turn-prefix feature that was removed in the 4.6 models.Paper authors (via Hacker News, stolen-thoughts.com) · Simon Willison's Weblog explains mechanism → Critical Thinking: Encrypted reasoning blocks from frontier API responses can be decoded and ported to different models, sessions, and users, which dramatically improves open models and leaks personal data when shared publicly.AINews host (unattributed editorial) · Latent Space explains mechanism → Critical Thinking: Even if labs hide reasoning, tool interfaces may re-expose it, since disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.AINews Twitter recap (attributed to @_can1357) · Latent Space gives example → The reasoning tokens that were revealed were clearly never intended for human consumption, as evidenced by a snippet of GPT-5.5 thinking about CSS in raw, fragmentary notes.Article author (via Hacker News) · Simon Willison's Weblog