Mao-Lin Luo, Zhe-Xu Wang, Zi-Hao Zhou, Bo Ye, Jian Zhao, Min-Ling Zhang, Tong Wei
Continual post-training is becoming a central paradigm for adapting vision-language models to evolving tasks. Recent studies report that reinforcement learning is less prone to forgetting than supervised fine-tuning, motivating the view that RL is inherently resistant to forgetting. However, this view remains insufficiently validated, as existing evidence is largely drawn from outdated or homogeneous benchmarks. We revisit this assumption by introducing MRCL, a Multimodal Reasoning Continual Learning benchmark built from recent and diverse multimodal reasoning tasks. Experiments on MRCL show that standard reinforcement learning still suffers from catastrophic forgetting during continual post-training. We trace the failure to an objective mismatch. The KL regularization used in common policy optimization methods is evaluated on current-task data, whereas forgetting is caused by behavioral drift on prior-task distributions. To address this problem, we propose Continual Policy Optimization (CPO), a replay-free method grounded in a prior-task behavioral KL objective. CPO derives a local Fisher surrogate from the historical KL objective and uses parameter movement as a gradient-free proxy for Fisher sensitivity, enabling sparse regularization with negligible additional overhead. Experiments on three model scales and comparisons with multiple RL baselines show that CPO consistently reduces forgetting while maintaining effective adaptation and preserving broader pretrained capabilities. The implementation code is available at https://github.com/MaolinLuo/CPO.
Junwei Yao, Zike Yang, Zhijun Wang, Yongxi Zhou, Wenbo Ye
LLM agents call tools that delete data or move money through the Model Context Protocol (MCP). The specification asks for a human who can deny tool calls but splits the mechanism between the server, which can annotate a tool or ask the user through the host (elicitation), and the host application, which decides whether to prompt. We measure both sides from source code. In a 1,000-repository sample of the official MCP registry, an estimated 90% of destructive tools (95% CI 82-95) have no server-side confirmation, and most servers that elicit proceed, or fall back to a flag the model can set, when the host cannot relay the question. Of 41 popular open-source hosts and agent SDKs, 32 have no approval step by default, including 16 of 25 applications. Pairing sampled destructive tools flagged by a rule-based extractor with these applications, 60.7% of pairs have no enforced human approval by default. Through four agent SDKs, four of five low-cost models executed an injected deletion in 479 of 960 runs. If hosts prompted for every tool not annotated read-only or non-destructive, the unapproved share of these pairs would fall to 5.4%; universal elicitation support would leave 56.7%. Applications should therefore configure approval.
Whole-prompt translation evaluates a request in different languages but does not compare which component carries a language. We introduce a paired reversal: translate the complete request component or the complete operation component into the same target language, holding behaviour, English frame and translated-span count fixed. Component identity and non-English text quantity change together. Assigning English, Chinese and Spanish independently to the two components yields nine configurations on 190 of 300 sampled harmful behaviours retained after consensus annotation and automatic gating. Refusal scored by language-model judges is the primary outcome; unsafe compliance is reported separately. On Gemini 2.5 Flash and DeepSeek, three refusal swaps are about 5.3 percentage points and the fourth is -0.5; none survives the pre-specified six-test Holm correction. In an exploratory extension, Mistral Small 4 has a 13.7-point Spanish swap (95% interval [8.4, 18.9]), which survives the twelve-test Holm sensitivity correction under the primary judge, and 12.1 points under a second judge. GPT-5.4-mini estimates are within 2 points of zero. Component assignment supplies an additional evaluation dimension. Evaluations should record component languages and translated text quantity, comparing matched behaviours at equal translated-span count while interpreting refusal and unsafe compliance separately.
Zhengxi Lu, Jiabo Ye, Fei Tang Nankai University, Yongliang Shen Zhejiang University, Haiyang Xu, Ziwei Zheng Xi'an Jiaotong University, Weiming Lu Zhejiang University, Ming Yan Alibaba Group 等 11 位作者
GUI agents have demonstrated remarkable progress in automating complex user interface interactions. However, training such agents for long-horizon tasks remains challenging. Single-turn reinforcement learning conditions on expert histories during training but self-generated histories during deployment, causing distribution mismatch. Online multi-turn methods eliminate this gap via environment interaction but suffer from sparse rewards and prohibitive costs. We propose \underline{\textbf{E}}xperience-driven \underline{\textbf{M}}ulti-turn \underline{\textbf{P}}olicy \underline{\textbf{O}}ptimization (\textbf{EMPO}), which leverages expert trajectories as environment experiences for on-policy multi-turn training. The agent constructs self-generated history throughout rollouts; when actions match expert experiences, the trajectory provides valid state transitions, and a Patch Module recovers mismatched steps to maintain on-policy rollouts. EMPO further incorporates discounted future rewards and dual-level advantage estimation to capture long-horizon dependencies. We also propose \textbf{AndroidControl-Real}, an evaluation metric strongly correlated with real-world performance (R^2=0.934). With only 1K public trajectories as RL experiences, our method achieves substantial gains over the base model (e.g., +12.0% on AndroidWorld and +23.8% on AITW) and achieves competitive performance against strong baselines such as UI-TARS-7B and GPT-4o, demonstrating better generalization than prior single-turn RL approaches. Code available: https://anonymous.4open.science/r/UI-S1-0DAF.
尚无 AI 解读，展示原文摘要
cs.CL
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表