Researchers at the University of Illinois and Amazon have developed a framework that ensembles the reasoning structures of multiple large language models rather than just their final answers.

The "Reasoning Consensus" method extracts Directed Acyclic Graphs (DAGs) from chain-of-thought reasoning and weights each step by how many independent traces support it. The approach addresses a key limitation in current LLM reasoning: users cannot assess which steps are well-supported or what alternatives the model considered.

Performance Gains Across Six Benchmarks

The framework outperformed majority-vote baselines across six datasets spanning statutory interpretation, graduate-level science, narrative reasoning, and first-order logic. The largest improvement was 3.1% accuracy on MuSR-MM, a narrative multi-hop reasoning benchmark.

On single-model applications, the method matched or exceeded self-consistency approaches while providing an inspectable consensus reasoning graph. This transparency allows users to see which reasoning steps have broad support versus those that rely on isolated chains of thought.

The research team tested the approach on tasks requiring complex multi-step reasoning where traditional ensemble methods that only combine final answers often fail to capture the nuanced reasoning process.

"The framework can also be used to analyze diverse reasoning perspectives for a problem," the authors wrote in their paper published on arXiv.

Ensemble weights correlated with LLM-judge rankings of reasoning quality at Spearman correlation coefficients between 0.30 and 0.51. In head-to-head comparisons, consensus subgraphs were preferred over alternatives leading to majority-vote answers in 54.4% to 65.4% of cases across five datasets.

The method works by parsing reasoning chains into structured graphs, identifying common reasoning patterns across multiple model runs, and weighting steps based on how frequently they appear in independent reasoning traces.

The research suggests a path toward more interpretable AI reasoning systems where users can inspect not just what models conclude, but how they reached those conclusions and which reasoning steps have the strongest support.