Extracting Search Trees from LLM Reasoning Traces Reveals Myopic Planning
Sixing Chen, Ji-An Li, Saner Cakir, Sinan Akcali, Kayla Lee, Marcelo G. Mattar
原文摘要
Large language models (LLMs), especially reasoning models, generate extended chain-of-thought (CoT) reasoning that often contains explicit deliberation over possible futures. Yet it remains unclear whether this deliberation constitutes genuine planning, how it is structured, and what aspects of it drive performance. Here we introduce a method for characterizing LLM planning by extracting search trees from reasoning traces in the board game four-in-a-row. By fitting computational models to the extracted search trees, we quantify how plans are structured and how they influence move decisions. We find that LLMs search less deeply than humans, and that their performance is better predicted by search breadth than depth. Most strikingly, although LLMs expand deep nodes in their traces, their move choices are best explained by a myopic model that ignores those nodes entirely. A causal intervention that selectively prunes CoT paragraphs supports that move selection depends predominantly on shallow rather than deep nodes. These patterns contrast with human planning, where performance is driven primarily by deep search. Together, our findings reveal a key difference between LLM and human planning: human expertise rests on deeper search, whereas LLMs generate deep lookahead but fail to act on it. This dissociation offers concrete targets for aligning LLM and human planning. More broadly, our framework provides a generalizable approach for interpreting the structure of LLM planning across strategic domains.
cs.AI