跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2605.25443 · v3

Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Models

首次提交 2026-05-25 13:42
版本更新 2026-11-14 19:47
雷达观测 2026-11-17 12:06
arXiv ↗PDF ↗

摘要

Multi-domain capabilities are critical for practically applicable Large Reasoning Models (LRMs). Therefore, mixed-domain Reinforcement learning (RL) has become a crucial process to jointly train a single LRM with mixed domain data, but it often induces capability trade-offs among different domains. Existing methods attribute this issue to competing cross-domain interactions, and mitigate them by isolating learning directions and balancing learning progress. However, our observation suggests that they also impede supportive knowledge sharing across domains, which make it difficult to match or surpass single-domain performance. To address this problem, we propose \textbf{M}ulti-domain \textbf{C}ontrastive \textbf{P}olicy \textbf{O}ptimization (MCPO), which applies contrastive learning to achieve diverse knowledge sharing and competition. We first localize the interaction units on critical reasoning segments, and then formulate mutually augmented examples to capture both positive and negative interactions. Subsequently, we design complementary contrastive objectives for cross-domain knowledge sharing and intra-domain knowledge consolidation. Overall, we utilize diverse cross-domain interactions to achieve compatibility across domains and discriminability within each domain to form a harmonious reasoning space. Experimental results across a broad range of domains show that MCPO significantly outperforms existing methods mitigating interactions over 3.17 average points, and even surpasses single-domain training over 1.05 average points.

作者备注:35 pages, 8 figures

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v3,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v3 · 2026-11-14 19:47 · 最新观测版本

Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Models

Multi-domain capabilities are critical for practically applicable Large Reasoning Models (LRMs). Therefore, mixed-domain Reinforcement learning (RL) has become a crucial process to jointly train a single LRM with mixed domain data, but it often induces capability trade-offs among different domains. Existing methods attribute this issue to competing cross-domain interactions, and mitigate them by isolating learning directions and balancing learning progress. However, our observation suggests that they also impede supportive knowledge sharing across domains, which make it difficult to match or surpass single-domain performance. To address this problem, we propose \textbf{M}ulti-domain \textbf{C}ontrastive \textbf{P}olicy \textbf{O}ptimization (MCPO), which applies contrastive learning to achieve diverse knowledge sharing and competition. We first localize the interaction units on critical reasoning segments, and then formulate mutually augmented examples to capture both positive and negative interactions. Subsequently, we design complementary contrastive objectives for cross-domain knowledge sharing and intra-domain knowledge consolidation. Overall, we utilize diverse cross-domain interactions to achieve compatibility across domains and discriminability within each domain to form a harmonious reasoning space. Experimental results across a broad range of domains show that MCPO significantly outperforms existing methods mitigating interactions over 3.17 average points, and even surpasses single-domain training over 1.05 average points.

查看此版本解读 →查看此版本原文 ↗