← 研踪 PaperTrailarXiv:2605.19256 · v2
Distribution Matching Distillation without Fake Score Network
首次提交 2026-05-19 10:05
版本更新 2026-11-05 12:55
雷达观测 2026-11-06 12:09
摘要
Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. Recent work combines DMD with a flow-map objective to exploit the benefits of both methods: mode-seeking and mode-covering behaviors. Since this fake-score network increases memory and update overhead, we study whether this explicit tracker can be avoided when the generator has a flow-map structure. In this work, we propose Fake-Score-network-Free DMD (FSF-DMD), a DMD-style correction for flow-map generators, replacing the auxiliary fake-score estimator with a generator-induced surrogate. We observe that this surrogate can be derived from the definition of the fake velocity under a certain perturbation assumption. Building on this insight, we derive a DMD-style objective without a fake-side network, extend it to backward simulation for flow maps, and to a self-teacher variant for training from scratch. In our ImageNet-1K $256\times 256$ experiments, FSF-DMD is competitive with the studied DMD baselines when initialized from pretrained networks and when trained from scratch.
作者备注：Accepted to NeurIPS 2026. 22 pages, 5 figures, 14 tables. Project page: https://snu-vgilab.github.io/FSF-DMD
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-05 12:55 · 最新观测版本
Distribution Matching Distillation without Fake Score Network
Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. Recent work combines DMD with a flow-map objective to exploit the benefits of both methods: mode-seeking and mode-covering behaviors. Since this fake-score network increases memory and update overhead, we study whether this explicit tracker can be avoided when the generator has a flow-map structure. In this work, we propose Fake-Score-network-Free DMD (FSF-DMD), a DMD-style correction for flow-map generators, replacing the auxiliary fake-score estimator with a generator-induced surrogate. We observe that this surrogate can be derived from the definition of the fake velocity under a certain perturbation assumption. Building on this insight, we derive a DMD-style objective without a fake-side network, extend it to backward simulation for flow maps, and to a self-teacher variant for training from scratch. In our ImageNet-1K $256\times 256$ experiments, FSF-DMD is competitive with the studied DMD baselines when initialized from pretrained networks and when trained from scratch.
查看此版本解读 →查看此版本原文 ↗