John Gruber's analysis of Anthropic's watermarking system reveals a significant gap between the company's initial promises and technical reality.

Anthropic originally claimed its watermarking would be "imperceptible" and wouldn't "change the meaning, quality, or readability" of Claude's responses. The company published a support document titled "How Claude Marks AI-Generated Content" that failed to explain the actual mechanism.

The real system works through semantic steganography. At each token generation point, Claude becomes more likely to select words from a dynamically generated "green" list over a "red" list. This bias creates a statistical fingerprint detectable only by Anthropic using their secret key.

How the Detection Works

The technique resembles analyzing coin flips for bias. Words are sorted into green and red categories deterministically at each generation step. Claude shows a slight preference for green-list words, creating patterns that become statistically significant over longer texts.

Detection confidence increases with text length. Short passages cannot be reliably identified as AI-generated, while longer texts show clearer statistical signatures. Only Anthropic can detect Claude's watermarks — other AI providers cannot identify each other's systems due to unique secret keys.

Gruber's core objection centers on semantic precision. He argues that "no two synonyms carry the exact same meaning" and that choosing "leaped" versus "jumped" represents a meaningful difference. The watermarking system inherently compromises word choice quality by introducing non-semantic selection criteria.

The controversy highlights tensions between regulatory compliance and output quality. Anthropic's EU regulation compliance requires watermarking, but the implementation directly contradicts their original transparency claims about preserving text quality.

James Padolsey's interactive essay "How AI Text Watermarking Works" provides the clearest technical explanation of these semantic watermarking techniques, demonstrating the statistical principles through hands-on examples.

Anthropic has not responded to criticism about the discrepancy between their marketing claims and the actual technical implementation.