← 研踪 PaperTrailarXiv:2605.26893 · v2
Learning Faithful Reasoning Processes with Spatio-Temporal Latent Dynamics for Large Language Models
首次提交 2026-05-26 19:54
版本更新 2026-10-26 20:02
雷达观测 2026-10-29 00:36
摘要
Chain-of-Thought (CoT) reasoning has significantly enhanced the reasoning capabilities of large language models (LLMs). However, existing reinforcement learning methods primarily optimize outcome-level rewards, providing limited supervision over the reasoning process. As a result, models may produce shortcut-prone or unstable reasoning trajectories even when the final answer is correct. We propose GeoFaith, a spatio-temporal latent dynamics framework for learning faithful reasoning processes without process reward models. GeoFaith combines final-state latent uncertainty, which measures whether a completed trajectory lies in a well-supported manifold region, with predictive-entropy dynamics, which penalize abnormal uncertainty patterns such as premature collapse, oscillation, and prolonged uncertainty. These signals are incorporated as process regularization into GRPO, jointly optimizing answer correctness and reasoning quality without step-level process annotations. Experiments show that GeoFaith consistently improves reasoning accuracy, produces more efficient reasoning trajectories, and outperforms both outcome-only reinforcement learning and process-supervision baselines.
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-10-26 20:02 · 最新观测版本
Learning Faithful Reasoning Processes with Spatio-Temporal Latent Dynamics for Large Language Models
Chain-of-Thought (CoT) reasoning has significantly enhanced the reasoning capabilities of large language models (LLMs). However, existing reinforcement learning methods primarily optimize outcome-level rewards, providing limited supervision over the reasoning process. As a result, models may produce shortcut-prone or unstable reasoning trajectories even when the final answer is correct. We propose GeoFaith, a spatio-temporal latent dynamics framework for learning faithful reasoning processes without process reward models. GeoFaith combines final-state latent uncertainty, which measures whether a completed trajectory lies in a well-supported manifold region, with predictive-entropy dynamics, which penalize abnormal uncertainty patterns such as premature collapse, oscillation, and prolonged uncertainty. These signals are incorporated as process regularization into GRPO, jointly optimizing answer correctness and reasoning quality without step-level process annotations. Experiments show that GeoFaith consistently improves reasoning accuracy, produces more efficient reasoning trajectories, and outperforms both outcome-only reinforcement learning and process-supervision baselines.
查看此版本解读 →查看此版本原文 ↗