← 研踪 PaperTrailarXiv:2611.12852 · v1
What kills $v$-prediction? A Patch-wise PCA Perspective on Pixel-Space Flow Matching
是什么扼杀了 v-prediction？像素空间流匹配的逐块 PCA 视角
首次提交 2026-11-10 02:44
版本更新 2026-11-10 02:44
雷达观测 2026-11-10 15:22
摘要
Latent Diffusion Models dominate generative modeling but face two-stage training complexity and the inherent reconstruction limits of pre-trained autoencoders. Pixel-space Flow Matching offers a simplified end-to-end alternative yet often encounters training difficulties when using the standard -prediction objective. Recent research attributes this difficulty to the off-manifold nature of noised targets, motivating a shift toward -prediction. In this paper, we investigate this explanation through patch-wise Principal Component Analysis (PCA) and identify a bandwidth bottleneck. The velocity in minor components is largely determined by the raw noisy input, requiring deep backbones to preserve information that competes with principal-component modeling. Through controlled experiments and representation analysis, we further examine why -prediction remains effective: its parameterization provides a raw-input bypass, while its learned input representation can spontaneously concentrate on principal directions. Under low-rank observations, our analysis of the induced flow characterizes -prediction as an implicit conditional-mean decoder, reconstructing discarded components from the retained information. These observations motivate the Raw-injected Transformer (RiT), which combines a lightweight raw-conditioned decoder with an output-side shortcut to separate semantic modeling from raw-state propagation. Our method restores effective -prediction and achieves competitive performance on ImageNet , providing a perspective on the interaction between prediction objectives and architectural bandwidth.
作者备注：Accepted as a Spotlight paper at NeurIPS 2026. Our code is available at https://github.com/THUMAI-Lab/RiT
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-10 02:44 · 最新观测版本
What kills $v$-prediction? A Patch-wise PCA Perspective on Pixel-Space Flow Matching
Latent Diffusion Models dominate generative modeling but face two-stage training complexity and the inherent reconstruction limits of pre-trained autoencoders. Pixel-space Flow Matching offers a simplified end-to-end alternative yet often encounters training difficulties when using the standard -prediction objective. Recent research attributes this difficulty to the off-manifold nature of noised targets, motivating a shift toward -prediction. In this paper, we investigate this explanation through patch-wise Principal Component Analysis (PCA) and identify a bandwidth bottleneck. The velocity in minor components is largely determined by the raw noisy input, requiring deep backbones to preserve information that competes with principal-component modeling. Through controlled experiments and representation analysis, we further examine why -prediction remains effective: its parameterization provides a raw-input bypass, while its learned input representation can spontaneously concentrate on principal directions. Under low-rank observations, our analysis of the induced flow characterizes -prediction as an implicit conditional-mean decoder, reconstructing discarded components from the retained information. These observations motivate the Raw-injected Transformer (RiT), which combines a lightweight raw-conditioned decoder with an output-side shortcut to separate semantic modeling from raw-state propagation. Our method restores effective -prediction and achieves competitive performance on ImageNet , providing a perspective on the interaction between prediction objectives and architectural bandwidth.
查看此版本解读 →查看此版本原文 ↗