Academic researchers have published a comprehensive audit questioning the validity of popular AI agent safety benchmarks, finding that widely-cited measurements may not reliably indicate actual safety.

The study, published on arXiv, examined four prominent benchmarks — R-Judge, InjecAgent, AgentHarm, and AgentDojo — testing them on up to 22 AI models including systems from major labs.

Fundamental scoring problems emerge

The researchers identified a critical flaw in how these benchmarks calculate safety scores. On R-Judge, an "always positive" policy that never discriminates between safe and unsafe responses achieves an F1 score of 0.690 — higher than five of the 21 models that actually attempt to make safety distinctions.

The three broad-coverage benchmarks ranked the same 18 models in completely different orders. When the research team tested correlations on smaller panels of seven models, they found wild swings from -0.64 to +0.02 between different safety measures.

A quarter of randomly selected seven-model subsets produced correlation coefficients above 0.5 in either direction, suggesting the rankings are largely statistical noise.

Capability versus safety trade-offs

The audit revealed a troubling inverse relationship between model capability and safety performance. Higher-capability models scored worse on misalignment safety measures, with a correlation of -0.44 across 21 models.

On a paired panel of 20 models, this capability-safety trade-off reached -1.00 with high statistical significance. The pattern held even when researchers controlled for different AI lab approaches using leave-one-organization-out analysis.

When expanded to 41 models, the misalignment correlation weakened to -0.16, though the researchers noted this change wasn't statistically significant.

AgentHarm shows strongest validity

Among the benchmarks tested, AgentHarm demonstrated the most robust correlation with other safety measures, achieving a 0.72 correlation with jailbreak safety after controlling for capability differences.

However, the researchers cautioned that both instruments measure "harmful compliance" rather than general safety, making this convergent validity rather than proof of comprehensive safety measurement.

The study concludes that safety claims require explicit specification of the benchmark, metric, target behavior, and model panel used — the current practice of citing safety scores interchangeably lacks scientific rigor.