MIT researchers have identified a critical weakness in current AI systems: their inability to handle ambiguous user requests effectively.

The study, published on arXiv, introduces "task alignment" as a fundamental challenge for AI assistants. Unlike existing benchmarks that test models on clearly defined tasks, real users often arrive with incomplete, exploratory, or contradictory goals.

Researchers Andy Dai, Zexue He, and colleagues formalized this problem as a Partially Observable Markov Decision Process (POMDP), where AI models must infer the user's true intent from limited and evolving information.

The team tested models across shopping, coding, and professional work scenarios. Results showed AI systems recover users' intended tasks only 22-32% of the time under ambiguous conditions.

Human participants in the same scenarios achieved 48% success rates, significantly outperforming all evaluated AI models.

Key failure modes identified

The research revealed three primary weaknesses in current AI systems:

  • Acting prematurely before understanding user intent
  • Ineffective interaction strategies for clarifying ambiguous requests
  • Failure to resolve uncertainty through follow-up questions

The researchers validated their findings through both computational experiments and human user studies, confirming that current models lack essential interaction abilities required for reliable AI agency.

Supervised fine-tuning and reinforcement learning showed some improvement in task alignment capabilities. However, even post-training enhancements failed to close the substantial gap between AI and human performance.

The study suggests that while AI models excel at executing well-defined tasks, they struggle fundamentally with the interpretive and clarification skills humans use naturally in ambiguous situations.

This research highlights a significant barrier to deploying AI assistants in real-world scenarios where user intent is rarely crystal clear from the outset.