A developer debugging Pi's code editor discovered that Anthropic's newest models perform worse at tool calling than their predecessors.

Claude Opus 4.8 and Sonnet 5 frequently generate malformed tool calls when using Pi's edit function, inventing extra fields like requireUnique, matchCase, and oldText2 that don't exist in the schema. The older Anthropic models don't exhibit this behavior.

The failures occur despite the core edit instructions being correct. The models produce accurate oldText and newText payloads but append nonsensical parameters that cause Pi to reject the tool call.

Training on forgiving systems backfires

The regression likely stems from post-training on Claude Code or similar harnesses that tolerate malformed tool calls. Anthropic's own client contains retry logic, parameter aliases, and filters for unknown keys — essentially absorbing errors silently.

When reinforcement learning occurs in such forgiving environments, slightly broken tool calls can still complete tasks and receive rewards. The model learns that extra parameters are acceptable, even beneficial.

This creates problems when the same model encounters stricter tool schemas. The developer found that enabling "strict tool invocation" eliminated the failures entirely, suggesting the model can follow schemas when forced to.

Implications for tool ecosystem

The findings raise concerns about model adaptability. Where Opus 4.5 adapted well to different tool shapes, newer models may struggle with schemas that differ from their training environment.

Alternative tool designs could become "off-distribution" for models trained primarily on one canonical format. Stronger priors from extensive post-training might actually make models less flexible, not more.

The issue appears context-dependent, occurring mainly in multi-turn conversations where models have read files and composed complex edits. Fresh single-turn prompts don't reproduce the problem.

Anthropichas not documented the exact tool schemas used in Claude Code's training, making it difficult for developers to predict compatibility issues with custom tools.