← 研踪 PaperTrailCVF · CVPR 2026:cvf:faebc81bb0795716d0f80cdca342bb1a · 记录 #1
The Drift Kernel: Why Diffusion Models Change Even When Told Not To
来源日期（仅到年份） 2026
来源记录日期 2026
雷达观测 2026-11-01 22:51
摘要
Even when told to "do nothing," modern diffusion models subtly alter their output relative to the input they are supposed to preserve. We call this effect No-Op Drift. We introduce the Drift Kernel K_M(sigma), defined as the expected perceptual deviation induced when running a diffusion model at noise strength sigma under a null instruction. Using 120,000 baseline samples (30,000 per model across SD15, SD21, SDXL, and InstructPix2Pix) and 9,600 ablation samples (four strengths, null versus strict copy prompts), we show that variance-driven diffusion models follow a quadratic form K_M(sigma) approximately equal to k_M sigma^2 plus c_M, with aggregate R^2 equal to 0.97. We derive this scaling from first principles via a Taylor expansion of the decoder, yielding k_M equal to Tr(J_D J_D^T), which depends only on the decoder Jacobian and not on prompts. To validate mechanistic structure, we construct synthetic decoders that reproduce the two regimes seen in practice: quadratic variance-driven drift and flat, high-variance edit-driven drift. We show that prompt wording has negligible effect (less than 17 percent coefficient difference), proving that drift is structural and not prompt-induced. We release NoOp-Bench, a benchmark with 10,000 inputs and full code for reproducible kernel estimation. Additional proofs, ablations, LPIPS and CLIP metrics, and extended visualizations appear in the Supplementary Material.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
The Drift Kernel: Why Diffusion Models Change Even When Told Not To
Even when told to "do nothing," modern diffusion models subtly alter their output relative to the input they are supposed to preserve. We call this effect No-Op Drift. We introduce the Drift Kernel K_M(sigma), defined as the expected perceptual deviation induced when running a diffusion model at noise strength sigma under a null instruction. Using 120,000 baseline samples (30,000 per model across SD15, SD21, SDXL, and InstructPix2Pix) and 9,600 ablation samples (four strengths, null versus strict copy prompts), we show that variance-driven diffusion models follow a quadratic form K_M(sigma) approximately equal to k_M sigma^2 plus c_M, with aggregate R^2 equal to 0.97. We derive this scaling from first principles via a Taylor expansion of the decoder, yielding k_M equal to Tr(J_D J_D^T), which depends only on the decoder Jacobian and not on prompts. To validate mechanistic structure, we construct synthetic decoders that reproduce the two regimes seen in practice: quadratic variance-driven drift and flat, high-variance edit-driven drift. We show that prompt wording has negligible effect (less than 17 percent coefficient difference), proving that drift is structural and not prompt-induced. We release NoOp-Bench, a benchmark with 10,000 inputs and full code for reproducible kernel estimation. Additional proofs, ablations, LPIPS and CLIP metrics, and extended visualizations appear in the Supplementary Material.
查看此版本解读 →查看此版本原文 ↗