Interactive LLM agents increasingly work with users across tasks, yet remembering a previous correction does not ensure that the agent changes its behavior in the future. In a diagnostic constructed from anonymized real-world agent interactions, agents using Mem0 still violate 57.5% of applicable preference checks. We introduce TellOnce, a drop-in personalization layer that turns conversational corrections into persistent, scoped, and revisable behavioral rules without updating the underlying agent model. TellOnce identifies which requirements should persist, determines when they apply, reconciles subsequent feedback with existing rules, and prepares reusable checks when reliable verification is possible. We evaluate whether learned corrections improve future behavior by freezing the learned state and testing on subsequent tasks without additional user corrections. Across three models on ClawArena and MemoryArena, TellOnce achieves the lowest preference violation rate in all six settings, reducing violations relative to PGMem by 19.3-42.3 percentage points on ClawArena and 3.0-22.9 points on MemoryArena, while maintaining broadly comparable task performance. On SWE-Together, TellOnce further improves task quality on new coding tasks by learning from feedback received during earlier tasks. Experiment code is available at https://github.com/YujunZhou/TellOnce_exp, and the deployable skill at https://github.com/YujunZhou/tellonce.
Zhenwen Liang Tencent AI Lab, Ruosen Li University of Texas at Dallas, Yujun Zhou University of Notre Dame, Linfeng Song Google, Dian Yu Tencent AI Lab, Xinya Du University of Texas at Dallas, Haitao Mi Tencent AI Lab, Dong Yu CapitalOne
Assessing the quality of Large Language Model (LLM) outputs becomes especially challenging in high-branching settings, where a single prompt yields many plausible candidates. Existing verifiers typically operate on the surface text (e.g., reward models, LLM judges, majority voting) or on confidence proxies derived from token probabilities, both of which can be brittle: the former can be influenced by stylistic artifacts, while the latter is often miscalibrated. In this paper, we study a third source of information—the model’s hidden states—for binary correctness verification in tasks with a reliable success/failure signal (e.g., deterministic checkers or reference-grounded answers). We find that correct and incorrect solutions exhibit measurable geometric differences in their hidden-state trajectories. To isolate this signal with minimal modeling assumptions, we introduce Clue (Clustering and Experience-based Verification), a training-free, non-parametric verifier. Clue summarizes each reasoning trace by an activation delta—the difference between hidden states at the start and end of the explicit reasoning span—and predicts correctness by comparing this delta to two class centroids computed from labeled experience. Across math (AIME 24/25), scientific QA (GPQA), and a multi-domain benchmark (WebInstruct-verified), Clue improves selection and reranking, with particularly strong gains on smaller or less-calibrated models. For example, on AIME 24 with a 1.5B model, Clue raises accuracy from 56.7% (majority@64) to 70.0% (top-maj@16).
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Zhenwen Liang Tencent AI Lab, Yujun Zhou University of Notre Dame, Sidi Lu ByteDance Seed, Xiangliang Zhang University of Notre Dame, Haitao Mi Tencent AI Lab, Dong Yu CapitalOne
Reinforcement Learning (RL) enhances LLM reasoning, yet a paradox emerges as models scale: strong base models saturate standard benchmarks (e.g., MATH), yielding correct but homogeneous solutions. In such environments, the lack of failure cases causes the advantage signal in group-relative algorithms (e.g., GRPO) to vanish, driving policies into mode collapse. To address this, we propose Constrained Uniform Top-K Sampling (CUTS), a parameter-free decoding strategy enforcing structure-preserving exploration. Unlike standard sampling that follows model biases, CUTS flattens the local optimization landscape by sampling uniformly from constrained high-confidence candidates. We integrate this into Mixed-CUTS, a training framework synergizing exploitative and exploratory rollouts to amplify intra-group advantage variance. Experiments on Qwen3 models demonstrate that our approach prevents policy degeneration and significantly boosts out-of-domain generalization. Notably, Mixed-CUTS improves Pass@1 accuracy on the challenging AIME25 benchmark by up to 15.1% over standard GRPO, validating that maintaining diversity within the semantic manifold is critical for rigorous reasoning.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · 2026 · 正式发表