跳到正文
研踪PaperTrail

论文库

检索计算机领域论文，按研究方向、作者或关键词筛选。

部分采集异常
正在读取账号与研究记录…

全站更新概况

+0新增论文
+0其中 AI 相关
+2已有论文修订
+100新增 AI 导读

arXiv 最近完成扫描：2026-11-15 09:49 · 全站最近入库：2026-11-14 15:09各信源进度 →

统计口径与数据新鲜度

统计 2026-11-14 10:39 — 2026-11-15 10:39（北京时间） · 点击数字查看对应简报；AI 相关包含在新增论文内。

最近完成检查：2026-11-15 09:49最近发现新增版本：2026-11-14 15:09库内论文最新更新时间：2026-11-13 02:59

每分钟刷新已保存的数据。空检查不代表采集异常；新收录可能包含历史论文。AI 导读按新增版本计数。

23,348 已收录论文19,580 AI 相关6,729 有修订112 HF 社区发现

浏览论文库

研究日报 / 周报 →按信源浏览 →

时间范围跟随排序：版本更新、本站收录或首次发表。最新收录可能包含历史论文。部分信源异常，查看状态 →

找到 3 篇论文 · 按版本更新时间排序上轮扫描起点 2026-11-15 09:48（北京时间）
版本更新 · v2arXiv:2606.27767

Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization

Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef Mroueh

原文摘要

Optimizing functionals over the space of probability measures is now ubiquitous in machine learning. A widely used approach is to perform the optimization directly over the Wasserstein space, but many objective functionals of practical interest are non-convex along Wasserstein geodesics, making the analysis of standard first-order methods challenging. In this work, we study a class of objectives over the Wasserstein space that admit a difference-of-convex (DC) decomposition and we lift the classical convex-concave procedure (CCCP) to this setting. Under smoothness and strong convexity assumptions on the convex components of the decomposition, we prove almost stationarity along the iterates of the resulting algorithm. Our main focus is on the Maximum Mean Discrepancy (MMD) and the Energy Distance (ED) functionals, for which we develop explicit Wasserstein DC decompositions, and establish local convergence of the scheme under mild assumptions. Empirically, we show that well-chosen DC decompositions yield faster and more stable convergence than Wasserstein gradient descent on these MMD objectives.

cs.LG · math.OC

查看论文
arXiv ↗PDF ↗
版本更新 · v6arXiv:2601.22478

Transformation-Augmented GRPO for Enhancing Exploration in Reasoning of Large Language Models

Khiem Le, Phuc Nguyen, Youssef Mroueh, Chi-Heng Lin, Shangqian Gao, Ting Hua, Nitesh V. Chawla

原文摘要

Group Relative Policy Optimization (GRPO) has become the dominant reinforcement learning method for eliciting reasoning capabilities in LLMs using verifiable rewards. Still, it suffers from two critical issues that limit models' exploration: \textbf{Gradient vanishing} and \textbf{Diversity collapse}. When a training question is too easy or too hard, all sampled responses receive identical rewards, yielding no gradient signal. Meanwhile, the model tends to collapse its responses toward a single reasoning pattern rather than exploring diverse strategies. We propose Transformation-Augmented GRPO (TA-GRPO), a simple yet effective method that addresses both issues via question transformation. For each training question, we automatically create multiple problem-equivalent rephrased questions that alter wording, format, and information order while preserving the underlying intent. As these rephrasings shift the model's perceived difficulty, responses across the original question and its rephrasings receive mixed rewards and encode more diverse reasoning strategies within it. TA-GRPO jointly computes advantages over this expanded response group and aligns all importance sampling ratios to the original question, enabling the model to learn from a richer set of attempts. Experiments with four LLMs (Qwen3-1.7B/4B-Base and Llama-3.2-1B/3B-Instruct) show that TA-GRPO consistently improves pass@$k$ on competition-level mathematics benchmarks (AMC, OlympiadBench, AIME24, AIME25) and out-of-distribution benchmarks (Minerva, GPQA-Diamond). Notably, it improves the average pass@32 of Qwen3-1.7B-Base and Qwen3-4B-Base by \textbf{4.97}\% and \textbf{4.35}\%, respectively, and matches the pass@32 of baselines trained with up to 2.5x more data.

cs.LG

查看论文
arXiv ↗PDF ↗
版本更新 · v2arXiv:2505.22257Google Research

重新审视组相对策略优化：在策略与离策略训练的洞见

Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training

Youssef Mroueh, Nicolas Dupuis, Brian Belgodere, Apoorva Nitsure, Mattia Rigotti, Kristjan Greenewald, Jiri Navratil, Jerret Ross 等 9 位作者

AI 速读 基于摘要

作者将 GRPO 扩展到离策略设定，并比较在策略与离策略两种 GRPO 变体在可验证奖励强化学习后训练中的表现。

cs.LG · stat.ML

arXiv ↗PDF ↗
查看原文摘要

We revisit Group Relative Policy Optimization (GRPO) in both on-policy and off-policy optimization regimes. Our motivation comes from recent work on off-policy Proximal Policy Optimization (PPO), which improves training stability, sampling efficiency, and memory usage. In addition, a recent analysis of GRPO suggests that estimating the advantage function with off-policy samples could be beneficial. Building on these observations, we adapt GRPO to the off-policy setting. We show that both on-policy and off-policy GRPO objectives yield an improvement in the reward. This result motivates the use of clipped surrogate objectives in the off-policy version of GRPO. We then compare the empirical performance of reinforcement learning with verifiable rewards in post-training using both GRPO variants. Our results show that off-policy GRPO either significantly outperforms or performs on par with its on-policy counterpart.