跳到正文
研踪PaperTrail
← 研踪 PaperTrail
正在查看已观测的 v1。查看最新 v2 →

arXiv:2611.13249 · v1

Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

Lin Yao
cs.LGcs.CL
首次提交 2026-11-10 05:21
版本更新 2026-11-10 05:21
雷达观测 2026-11-11 11:04
arXiv ↗PDF ↗

摘要

Multi-teacher on-policy distillation assigns each sample a domain teacher but typically updates shared student parameters. We introduce Domain-Routed Delta Optimization (DRD-MOPD), which specifies which parameter components each domain's samples update. The student is the sum of a frozen base and trainable components initialized from expert-minus-base weights. All components participate in every forward pass; four update modes define the correspondence between sample domains and components, each with independent optimizer states. We analyze these update rules and measure retention of the original expert geometry through angles, projection coefficients, and energy. Prior work such as Open-MOPD distills directly from the base model; 97.02% of its released final checkpoint's increment energy lies outside the span of the original expert directions. In comparison, our Own checkpoint after 20 updates retains coefficients near one along each component's original direction and pairwise angles ranging from 88.04 to 92.41 degrees; its exported BF16 student's increment has 4.90% outside energy, versus 10.52% for the sum of saved FP32 components. The same checkpoint achieves a 32.72% macro score across six benchmarks, compared with 32.44% for its initial task-arithmetic merge. These measurements describe geometry and capability at the evaluated training stages and settings.

查看 arXiv 原文 ↗阅读 PDF ↗

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存，登录后共享已有结果。

查看最新版本解读 →

版本差异

仅对比实际观测的元数据，不代表 PDF 正文或实验变化。版本可能不连续；带删除线为移除，下划线为新增。

摘要变化

Multi-teacher on-policy distillation assigns eachcan sampleyield auneven domain teacher but typically updates shared student parameterscapabilities. We introducepropose Domain-Routed Delta Optimization (DRD-MOPD), which specifies which parameter components each domain's samples update. The student is the sum ofcomposing a frozen base and trainable components initialized fromwith expert-minus-base weights. All components participate in everyinitialized forwardtrainable pass;deltas. fourDomains update modes define the correspondence between sample domains and components,their eachdeltas with independent optimizer states. We analyze these update rules and measure retention of the original expert geometry through angles, projection coefficients, and energyclipping. Prior workAfter suchjoint astraining, OpenPost-MOPD distills directlyseparate fromexports the base model; 97.02% of its released final checkpoint's increment energy lies outside the span of the original expert directions. In comparison, our Own checkpoint after 20 updates retains coefficients nearplus one along each component's original direction and pairwise angles ranging from 88.04trained todelta 92.41per degrees;domain, itsselected exportedby BF16known student'stask incrementdomain haswithout 4retraining.90% outsideUsing energyAdamW, versus 10.52% for the sum of saved FP32 components. The sameunweighted checkpointfull achievesstudent areaches 32.7283% macro scoreMacro across six benchmarks, comparedafter with20 updates, versus 32.4421% for itsa initialmatched task-arithmeticshared mergecontrol. These measurements describe geometry and capability at thePost-separate evaluatedraises trainingMacro stageswith andOpen-MOPD settingsweighting.

观测到的版本

相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-13 01:38 · 最新观测版本

Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

Multi-teacher on-policy distillation can yield uneven domain capabilities. We propose Domain-Routed Delta Optimization (DRD-MOPD), composing a frozen base with expert-initialized trainable deltas. Domains update their deltas with independent optimizer states and clipping. After joint training, Post-separate exports the base plus one trained delta per domain, selected by known task domain without retraining. Using AdamW, the unweighted full student reaches 32.83% Macro across six benchmarks after 20 updates, versus 32.21% for a matched shared control. Post-separate raises Macro with Open-MOPD weighting.

查看此版本解读 →查看此版本原文 ↗
v1 · 2026-11-10 05:21

Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

Multi-teacher on-policy distillation assigns each sample a domain teacher but typically updates shared student parameters. We introduce Domain-Routed Delta Optimization (DRD-MOPD), which specifies which parameter components each domain's samples update. The student is the sum of a frozen base and trainable components initialized from expert-minus-base weights. All components participate in every forward pass; four update modes define the correspondence between sample domains and components, each with independent optimizer states. We analyze these update rules and measure retention of the original expert geometry through angles, projection coefficients, and energy. Prior work such as Open-MOPD distills directly from the base model; 97.02% of its released final checkpoint's increment energy lies outside the span of the original expert directions. In comparison, our Own checkpoint after 20 updates retains coefficients near one along each component's original direction and pairwise angles ranging from 88.04 to 92.41 degrees; its exported BF16 student's increment has 4.90% outside energy, versus 10.52% for the sum of saved FP32 components. The same checkpoint achieves a 32.72% macro score across six benchmarks, compared with 32.44% for its initial task-arithmetic merge. These measurements describe geometry and capability at the evaluated training stages and settings.

查看此版本解读 →查看此版本原文 ↗