A comprehensive study of enterprise AI deployment has quantified why corporate AI programs stall at widely reported rates, finding that 44% of configurations fail to meet production standards despite appearing to work in demonstrations.

Researcher Prerit Ahuja tested six document-heavy workflows typical of regulated financial services across four model families and three tool configurations. The study produced 5,093 scored output elements across 72 total configurations, each run three times to measure consistency.

Two assessment bars emerged as critical. A "demonstration bar" required a single correct run on a single case — 57 of 72 configurations cleared this threshold. The "production bar" demanded sustained accuracy, reproducibility across repeats, verifiable attribution, and meaningful confidence signals. Only 32 configurations met these stricter requirements.

The human review bottleneck

The study's key finding centers on review burden rather than raw accuracy. AI tools that provide no confidence assessment require human review of 100% of their output, offering reviewers no basis for prioritizing which results to check first.

Adding source citations and confidence scoring reduced the review burden to 49% while maintaining error tolerance in 17 of 20 configurations tested. A self-verification pass — where the AI double-checks its own work — cut review needs to 44% but increased latency by 2.3 times and was the only approach that failed to maintain acceptable error rates.

The research suggests enterprise AI value depends less on accuracy rates than on how much human oversight remains necessary. "The practical implication is that the value of an AI workflow is set less by how often it is right than by how much of it a human must still check," the paper states.

Measuring what matters

The study addresses a gap in enterprise AI evaluation, where most assessments focus on demonstration capability rather than production readiness. The "checking problem" — determining what must be verified before AI ships in regulated environments — has received little systematic measurement despite being widely cited as a deployment barrier.

Financial services firms face particular challenges given regulatory requirements for auditability and error accountability. The research provides a framework for measuring review burden that could apply across regulated industries where AI adoption has lagged despite strong demonstration results.