跳到正文
研踪PaperTrail

论文库

检索计算机领域论文，按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+2已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-15 13:55 · 全站最近入库：2026-11-14 15:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-14 14:46 — 2026-11-15 14:46（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-15 13:55最近发现新增版本：2026-11-14 15:09库内论文最新更新时间：2026-11-13 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

23,348 已收录论文19,580 AI 相关6,729 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 3 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-15 13:54（北京时间）
新论文 · v1arXiv:2611.08936

Ordered Policy Optimization

Dongkyu Derek Cho, Pan Xu

原文摘要

Proximal Policy Optimization (PPO) stabilizes policy updates by clipping the importance ratio between the new policy and the data-collecting policy. The standard rule uses one fixed clipping range for all samples. While this choice offers simplicity and effectiveness, it fails to distinguish statistically harmful extreme ratio values from large, yet tolerable changes. We study how to adapt the clipping radius to the reliability of the induced importance ratio distribution. Our case study demonstrates that reliable updates require controlling these extreme ratios, rather than merely verifying that individual ratios fall within a fixed interval. Motivated by this observation, we propose Ordered Policy Optimization (\textsc{OPO}). In each minibatch, \textsc{OPO} sorts the importance ratios, selects the extreme ratios that potentially have the largest effect on the update, and assigns them adaptive sample-wise radii instead of using the fixed clipping radius. These radii are derived from a prescribed tail profile, which specifies desired target values for the selected extreme ratios. The resulting objective remains first-order and requires only minibatch sorting and a modified policy loss. Experiments conducted on MuJoCo and Atari benchmarks demonstrate that this simple modification achieves strong performance relative to established baselines.

cs.LG · cs.AI · cs.RO · stat.ML

查看论文
arXiv ↗PDF ↗
版本更新 · v5arXiv:2503.09508

Gain-Sharing Optimization in Randomized Primal--Dual Analysis: Structure and Certification

Pan Xu

原文摘要

Randomized primal--dual guarantees for online matching and allocation often depend on a gain-sharing function that determines how algorithmic gains are distributed among dual variables and, in some settings, also affects allocation decisions. We study the best certificate obtainable by optimizing this function within a specified algorithm--analysis template. For an admissible function class F, we formulate the problem as the functional optimization $Γ(F)=\sup_{f\in F} L[f]$, where $L[f]$ is the worst-case expected dual-feasibility factor. We develop two complementary certification methods: auxiliary linear programs with explicit finite-grid error bounds, and a continuous approach based on monotone Volterra comparison and residual-based numerical verification. We apply them to the vertex-weighted Stochastic Balance template for online matching with stochastic rewards under uniform vanishing success probabilities. For the configuration-LP benchmark and specified dual updates, we optimize over the full admissible class $F_0$ of continuous non-decreasing functions from $[0,\infty)$ to $[0,1]$, and prove $Γ(F_0)\in[0.5802,0.5805]$. We also show that the more tractable subclass used for auxiliary-LP certification attains the same optimum.

cs.DS

查看论文
arXiv ↗PDF ↗
首次观测 · 记录 #1ACL Anthology · ACL 2026:acl:2026.acl-long.69ACL Anthology · ACL 2026

Reinforcement Learning for Self-Improving Agent with Skill Library

Jiongxiao Wang University of Wisconsin - Madison, Qiaojing Yan Amazon, Yawei Wang Amazon, Yijun Tian, Soumya Smruti Mishra Amazon, Zhichao Xu Amazon, Megha Gandhi Amazon, Panpan Xu Amazon 等 9 位作者

原文摘要

Large Language Model (LLM)-based agents have demonstrated remarkable capabilities in complex reasoning and multi-turn interactions but struggle to continuously improve and adapt when deployed in new environments. One promising approach is implementing skill libraries that allow agents to learn, validate, and apply new skills. However, current skill library approaches rely primarily on LLM prompting, making consistent skill library implementation challenging. To overcome these challenges, we propose a Reinforcement Learning (RL)-based approach to enhance agents’ self-improvement capabilities with a skill library. Specifically, we introduce Skill Augmented GRPO for self-Evolution (SAGE), a novel RL framework that systematically incorporates skills into learning. The framework’s key component, Sequential Rollout, iteratively deploys agents across a chain of similar tasks for each rollout. As agents navigate through the task chain, skills generated from previous tasks accumulate in the library and become available for subsequent tasks. Additionally, the framework enhances skill generation and utilization through a Skill-integrated Reward that complements the original outcome-based rewards. Experimental results on AppWorld demonstrate that SAGE, when applied to supervised-finetuned model with expert experience, achieves 8.9% higher Scenario Goal Completion while requiring 26% fewer interaction steps and generating 59% fewer tokens, substantially outperforming existing approaches in both accuracy and efficiency. Our code is available at https://github.com/amazon-science/SAGE.

cs.CL

Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · 2026 · 正式发表