Yuheng Zhang, Wenlin Yao, Changlong Yu, Yao Liu, Qingyu Yin, Bing Yin, Hyokun Yun, Lihong Li
Group relative policy optimization (GRPO) trains large language models with verifiable rewards by normalizing rewards within response groups. When every response in a group receives the same reward, the advantages vanish and the group provides no policy gradient signal. We propose Adaptive Rollout and Response Reuse Policy Optimization (AR3PO) to obtain informative groups with fewer generations. Adaptive rollout samples responses to each prompt in stages and stops sampling once the model generates a correct response. If all sampled responses are incorrect, response reuse replaces one of them with a past correct response to the same prompt, when available, and recomputes the group advantages. Policy-recomputed reuse trains on the updated groups using probabilities recomputed under the rollout policy; advantage-only reuse uses historical responses only to compute advantages. Across four mathematical benchmarks with 7B and 8B models, AR3PO consistently improves over GRPO and achieves performance comparable to DAPO while requiring up to $4.2\times$ fewer generated responses. Experiments with a 32B model also show comparable performance to DAPO at a similar training stage with substantially fewer generated responses.
Jiaming Leng, Yunying Bi University of Science and Technology of China, Chuan Qin Chinese Academy of Sciences, Zhenya Huang University of Science and Technology of China, Bing Yin iFLYTEK, Haojie Ren, Yanyong Zhang University of Science and Technology of China, Chao Wang University of Science and Technology of China
Urban transportation systems require precise modeling of dynamic spatiotemporal patterns across diverse tasks, such as traffic forecasting, electric vehicle (EV) charging demand prediction, and taxi dispatch. Existing approaches suffer from two key limitations: traditional deep learning models are task-specific and lack generalization capabilities, whereas Large Language Models (LLMs) struggle with structured spatiotemporal data and numerical reasoning. To bridge this gap, we propose TransLLM, a unified multi-task framework that synergizes spatiotemporal encoding with LLM reasoning through learnable prompt composition. To enable LLMs to perceive complex graph dependencies, we design a noise-augmented spatiotemporal encoder that projects structured signals into the LLM’s embedding space. Furthermore, to overcome the rigidity of fixed prompt templates in heterogeneous traffic scenarios, we introduce an instance-level prompt routing mechanism trained via reinforcement learning. The framework operates by encoding spatiotemporal patterns into contextual representations, dynamically composing personalized prompts to guide LLM reasoning, and projecting the resulting representations through specialized output layers to generate task-specific predictions. Experiments on seven datasets and three tasks demonstrate that TransLLM outperforms many baselines, showing superior adaptability in both supervised and zero-shot settings with excellent generalization and robustness. Our code and data are available at https://github.com/lengjiaming/TransLLM.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表
Ming Li, Pei Chen Amazon, Zhenhao Zhang Amazon, Tao Yang Amazon, Xinyang Zhang Anthropic, Han Li Shopify, Tianyu Cao, Ming Zeng Amazon 等 13 位作者
Large Language Models demonstrate strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC), a degradation in performance as information is revealed progressively in multi-turn settings. Motivated by the current progress on Reinforcement Learning with Verifiable Rewards (RLVR), we propose Curriculum Reinforcement Learning with Verifiable Accuracy and Abstention Rewards (RLAAR), a framework that encourages models not only to generate correct answers, but also to judge the solvability of questions in the multi-turn conversation setting. Our approach employs a competence-gated curriculum that incrementally increases dialogue difficulty (in terms of instruction shards), stabilizing training while promoting reliability. Using multi-turn, on-policy rollouts and a mixed-reward system, RLAAR teaches models to balance problem-solving with informed abstention, reducing premature answering behaviors that cause LiC. Evaluated on LiC benchmarks, RLAAR significantly mitigates LiC performance decay (62.6% to 75.1%) and improves calibrated abstention rates (33.5% to 73.4%). Together, these results provide a practical recipe for building multi-turn reliable and trustworthy LLMs.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表