Researchers have developed CUDAnalyst, a new analysis framework that decodes how large language models make planning decisions when generating CUDA kernel code as self-evolving agents.

The work, accepted at ICML 2026, addresses a critical opacity in current AI systems: how agents combine different types of feedback to make planning decisions across multiple generations of code.

Lead author Yee Hin Chong and colleagues from an unnamed institution found that standard evaluation methods fail to isolate feedback effects because iterative planning amplifies early errors and mixes feedback impacts with trajectory drift.

How CUDAnalyst Works

The framework uses trajectory freezing and selective feedback injection to provide controlled, generation-level attribution of planning decisions to specific feedback components.

This approach enables what the researchers call "principled coalitional-style attribution" - essentially showing which feedback signals matter most and how they interact during the planning process.

The system allows researchers to evaluate each generation independently while maintaining stable conditions across experiments.

Key Findings

The research revealed three major insights about AI agent planning behavior. Explicit planning only helps when feedback signals are properly aligned with the task objectives.

Effective planning emerges from structured interactions between multiple feedback sources rather than single feedback loops. High-level plans developed by stronger reasoning models can partially transfer to weaker models, suggesting a form of knowledge distillation in planning capabilities.

These patterns held consistent across different model backbones, representative workloads, and training approaches, indicating robust underlying principles.

The findings have implications for improving AI agent architectures, particularly in code generation tasks where multiple feedback signals - compilation errors, performance metrics, and correctness checks - must be balanced.

The research provides a methodological foundation for understanding and improving how AI agents process feedback in iterative planning scenarios, with potential applications beyond CUDA kernel generation to other complex reasoning tasks.