A new study reveals that access to AI advice dramatically reduces people's willingness to admit uncertainty, even when the AI guidance is incorrect and users are paid for accurate responses.

Researchers from three European universities conducted five experiments with 3,132 participants who answered difficult questions while having the option to decline responding. The team deliberately engineered scenarios where AI advice was wrong, separating AI usage from accuracy.

Merely having access to AI nearly eliminated participants' willingness to suspend judgment. This effect held whether users actively requested the advice or it was simply displayed on screen.

The results were stark. Participants answered more questions when AI was available but were correct only about one-third as often compared to when AI was unavailable. Despite this dramatic drop in accuracy, their confidence nearly doubled.

Financial incentives barely helped

Even when researchers offered financial rewards for correct answers and penalties for wrong ones, participants still sought and followed AI advice. The incentives led to slightly more accurate responses and increased willingness to say "I don't know," but far less than when AI was completely unavailable.

The study, published on arXiv, suggests AI suggestions may fundamentally alter human metacognition — the process by which people evaluate whether they know enough to answer a question.

"As AI suggestions grow ubiquitous and unsolicited, they may not simply affect answer accuracy; they may even alter the metacognitive threshold at which people decide whether they know enough to answer," the researchers wrote.

The findings raise concerns about AI systems that provide confident-sounding responses to nearly any query. Popular AI assistants like OpenAI's ChatGPT and Anthropic's Claude are designed to generate fluent answers even when uncertain.

The research comes as AI advice becomes increasingly embedded in search engines, productivity tools, and decision-making systems across industries. The study's authors warn that widespread AI deployment could erode a fundamental aspect of human judgment — knowing when to acknowledge the limits of one's knowledge.