Article · 2026-08-12 · 6 moments

[AINews] How to steal a Reasoning Trace

Speculative Decoding by any other name would distil as sweet ✦ AI generated

GUEST AINews host (unattributed editorial)GUEST AINews Twitter recap (attributed to @_can1357)GUEST AINews Twitter recap (attributed to @vipulved)GUEST AINews Twitter recap (attributed to @BlackHC)HOST Alessio Fanelli HOST Shawn Wang (swyx)

timeline · colored by role

Claim◆

Critical Thinking: Encrypted reasoning blocks from frontier API responses can be decoded and ported to different models, sessions, and users, which dramatically improves open models and leaks personal data when shared publicly.

The paper breaks the assumption that lab reasoning traces are secure. It shows encrypted reasoning blocks can be decoded and replayed, ported to weaker models to transcribe, improving open models — and leaking personal data from publicly shared traces.

transcript

AINews host (unattributed editorial): Since the o1 launch, frontier lab reasoning models have obscured their traces, with cryptographic signatures, for fear of distillation... Today's paper demonstrates that it is possible to DECODE and port these encrypted thoughts to different models/sessions/users… and to dramatically improve open models as a result. The alarming note is here: 'Further, if you ever shared online a Claude Code/Codex session with encrypted reasoning blobs, they can be decoded and leak your personal data. We did a preliminary scan of ~7,000 public traces and found 62 unique API keys, 33 email addresses, 33 passwords, and other sensitive data.'

explains mechanism · 3extends · 1

Mechanism

Critical Thinking: The attack reuses a valid encrypted reasoning block by replaying it into a different request, placing it in an assistant/model turn, and prompting a weaker model to transcribe the attached reasoning.

The mechanism is fully described: obtain a legitimate encrypted reasoning block, replay it into another request to a weaker model from the same provider, prefill the model to transcribe the reasoning, sample repeatedly, discard refusals, and reconcile noisy transcriptions.

transcript

AINews host (unattributed editorial): The technique is somewhat described in the paper: Obtain a legitimate encrypted/signed reasoning block from an API response. Replay that block into a different request—potentially another account/session—to a weaker model from the same provider. Place it in an assistant/model turn and prompt or prefill the weaker model to transcribe the attached reasoning. Sample repeatedly, discard refusals, and optionally reconcile multiple noisy transcriptions.

explains mechanism · 1extends · 1

Mechanism

Critical Thinking: The attack generalizes across model providers with concrete per-model templates, including bypassing an apparent ~50-token verbatim-output threshold via chunked continuations.

Concrete templates vary per model: for Claude, replay the signed block to Haiku 4.5; for GPT, inject the encrypted_content reasoning item and sample up to 50 outputs while bypassing the ~50-token verbatim threshold with chunked continuations; for Gemini, attach thought_signature with a <thought> prefill.

transcript

AINews host (unattributed editorial): The paper gives concrete templates with some minor variations per model: Claude: replay the signed thinking block to Haiku 4.5, followed by an assistant prefill such as <thinking-copy>. GPT: inject the encrypted_content reasoning item multiple times into a fabricated conversation; sample up to 50 outputs. It also describes bypassing an apparent ~50-token verbatim-output threshold using chunked continuations. Gemini: attach thought_signature to a model turn with a <thought> prefill, then use repeated sampling and reconciliation.

extends · 3

Claim

Critical Thinking: Even if labs hide reasoning, tool interfaces may re-expose it, since disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.

A practical corollary to the trace exposure: hiding reasoning is not robust, because tool interfaces such as a deep_think tool can still induce internal-format chain-of-thought output even when explicit thinking is disabled.

transcript

AINews Twitter recap (attributed to @_can1357): A practical corollary: even if labs hide reasoning, tool interfaces may re-expose it; @_can1357 notes that disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.

explains mechanism · 1

Context

Critical Thinking: The trace-exposure vulnerability does not imply practical mass theft of chain-of-thought for model training; it is more a stateless distributed-inference protocol optimization than a confidentiality barrier.

The technical discussion splits between a serious privacy/safety problem and a non-scalable distillation path; the encryption is framed as a stateless distributed-inference protocol optimization rather than a hard confidentiality barrier.

transcript

AINews Twitter recap (attributed to @vipulved): Discussion split between 'serious privacy/safety problem' and 'not a scalable distillation path.' @vipulved argues the attack does not imply practical mass theft of chain-of-thought for model training, framing the encryption more as a stateless distributed-inference protocol optimization than a hard confidentiality barrier.

rebuts · 3

Example◆

Critical Thinking: The practical corollaries of this episode are that public trace sharing is risky, hidden CoT is not a reliable monitoring interface, and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces.

Beyond the specifics, the episode sharpens broader lessons: public trace sharing is risky, hidden chain-of-thought is not a reliable monitoring interface, and labs may need stronger sandboxing, telemetry, and tool-surface guarantees.

transcript

AINews Twitter recap (attributed to @BlackHC): Still, the episode sharpens a few points: public trace sharing is risky; hidden CoT is not a reliable monitoring interface; and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces @BlackHC.

Highlight slides

Encrypted Reasoning Traces Can Be Decoded and Ported✦ from: Critical Thinking: Encrypted reasoning blocks from frontier API responses can be decoded and ported to different models, sessions, and users, which dramatically improves open models and leaks personal data when shared publicly.Public Traces Leak Personal Data✦ from: Critical Thinking: Encrypted reasoning blocks from frontier API responses can be decoded and ported to different models, sessions, and users, which dramatically improves open models and leaks personal data when shared publicly.Critical Thinking: What This Episode Sharpens✦ from: Critical Thinking: The practical corollaries of this episode are that public trace sharing is risky, hidden CoT is not a reliable monitoring interface, and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces.

Related episodes

Article · Simon Willison's Weblog

Critical Thinking: Encrypted reasoning blocks from frontier API responses can be decoded and ported to different models, sessions, and users, which dramatically improves open models and leaks personal data when shared publicly.

Critical Thinking: The attack reuses a valid encrypted reasoning block by replaying it into a different request, placing it in an assistant/model turn, and prompting a weaker model to transcribe the attached reasoning.

Critical Thinking: The attack generalizes across model providers with concrete per-model templates, including bypassing an apparent ~50-token verbatim-output threshold via chunked continuations.

Critical Thinking: Even if labs hide reasoning, tool interfaces may re-expose it, since disabling explicit thinking while providing a deep_think tool can still induce internal-format CoT output.

Critical Thinking: The trace-exposure vulnerability does not imply practical mass theft of chain-of-thought for model training; it is more a stateless distributed-inference protocol optimization than a confidentiality barrier.

Critical Thinking: The practical corollaries of this episode are that public trace sharing is risky, hidden CoT is not a reliable monitoring interface, and labs may need stronger guarantees around sandboxing, telemetry, and tool surfaces.

How I tricked Claude into leaking your deepest, darkest secrets

Auto mode is now the default in Claude Code for Pro, Max, and Team plans

Exploring the J-Space