← 研踪 PaperTrailarXiv:2609.38829 · v2
Diversity Combining for Multi-Path LLM Reasoning
首次提交 2026-09-30 10:52
版本更新 2026-10-27 05:03
雷达观测 2026-10-29 00:35
摘要
Multi-path reasoning methods such as self-consistency (SC) sample $K$ reasoning paths and choose the most frequent answer. However, their gains quickly plateau as $K$ increases, and locating this saturation point currently requires fitting a scaling model to accuracies measured at several values of $K$ or stopping online per query. We formalize multi-path LLM reasoning as a diversity combining problem from wireless communications: each path is a noisy channel observation, and the pairwise correlation of path correctness caps the design-effect effective sample size of the vote at a finite ceiling. Generalized least squares (GLS) analysis shows that, for paths with equal gains, the optimal linear combiner of latent embeddings is uniform exactly when every row of the path covariance has the same sum, as under exchangeability, supporting majority vote as the natural default in standard SC while leaving room for weighting or pruning under heterogeneous prompt-template branches. Across 5 models and 12 benchmarks, prompt-template diversity reduces path correlation in $55$ of $60$ model-benchmark cells, with a mean relative reduction of $16\%$ and reductions in every task domain. We derive an Adaptive-K rule that uses a four-path pilot to select $K^*$, retaining $96$--$103\%$ of MV@$K{=}32$ accuracy across Math, QA, and NLU.
作者备注：Accepted by NeurIPS 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-10-27 05:03 · 最新观测版本
Diversity Combining for Multi-Path LLM Reasoning
Multi-path reasoning methods such as self-consistency (SC) sample $K$ reasoning paths and choose the most frequent answer. However, their gains quickly plateau as $K$ increases, and locating this saturation point currently requires fitting a scaling model to accuracies measured at several values of $K$ or stopping online per query. We formalize multi-path LLM reasoning as a diversity combining problem from wireless communications: each path is a noisy channel observation, and the pairwise correlation of path correctness caps the design-effect effective sample size of the vote at a finite ceiling. Generalized least squares (GLS) analysis shows that, for paths with equal gains, the optimal linear combiner of latent embeddings is uniform exactly when every row of the path covariance has the same sum, as under exchangeability, supporting majority vote as the natural default in standard SC while leaving room for weighting or pruning under heterogeneous prompt-template branches. Across 5 models and 12 benchmarks, prompt-template diversity reduces path correlation in $55$ of $60$ model-benchmark cells, with a mean relative reduction of $16\%$ and reductions in every task domain. We derive an Adaptive-K rule that uses a four-path pilot to select $K^*$, retaining $96$--$103\%$ of MV@$K{=}32$ accuracy across Math, QA, and NLU.
查看此版本解读 →查看此版本原文 ↗