Claim◆Article
It is specifically the newest, state-of-the-art Claude models — Opus 4.8 and Sonnet 5 — that show this malformed-tool-call problem, while none of the older models do, meaning the SOTA models are worse at this particular tool schema than their older siblings.
Armin was surprised that the schema mistake gets worse, not better, with newer and more capable Claude models, since malformed tool calls are normally expected from weaker models rather than the SOTA ones. ✦ AI generated
Armin Ronacher · Simon Willison's Weblog · 2026-07-04 · original ↗
What surprised me is that this is getting worse with newer Anthropic models as both Opus 4.8 and Sonnet 5 show it but none of the older models. In other words, the SOTA models of the family are worse at this specific tool schema than their older siblings.
Read full article ↗excerpt · fair-use quotation
- ·Opus 4.8 and Sonnet 5 show malformed tool calls
- ·No older Claude models exhibit this issue
- ·Newest, most capable models are worse here
- ·Problem worsens with newer model generations
- ·Malformed calls expected from weaker models, not SOTA
- ·Older siblings handle this tool schema better
- ·Capability gains didn't fix this specific schema issue
Around this claim
This moment responds to
extends → Newer Claude models — including the flagship Opus 4.8, not just small models — sometimes call Pi's edit tool with extra, invented fields in the nested edits[] array, so the arguments fail to match the schema and Pi rejects the tool call and asks for a retry.Armin Ronacher · Simon Willison's Weblogextends → Newer Anthropic models have been specifically trained, presumably via reinforcement learning, to better use the edit tools baked into Claude Code, and this makes them more likely to misuse the differently-shaped custom edit tools of other coding harnesses like Pi.Armin Ronacher · Simon Willison's Weblogsupports → Newer Anthropic models have been specifically trained, presumably via reinforcement learning, to better use the edit tools baked into Claude Code, and this makes them more likely to misuse the differently-shaped custom edit tools of other coding harnesses like Pi.Armin Ronacher · Simon Willison's Weblog