← 研踪 PaperTrailarXiv:2609.31620 · v3
Rethinking the Reconstructio-Generation Gap in Representation Autoencoders
首次提交 2026-09-26 01:59
版本更新 2026-11-17 19:52
雷达观测 2026-11-18 11:33
摘要
Fusing shallower encoder layers into a Representation Autoencoder (RAE) sharpens pixel reconstruction but degrades diffusion generation. Prior studies and the research community commonly attribute this trade-off to the latent content itself, assuming detail-rich features are harder to model. We show that this gap stems from geometric mismatch at the encoder-decoder interface. Freezing a foundation encoder preserves rich visual information, but imposes an encoder-native Euclidean metric shaped without pixel decoding, whereas the decoder amplifies sampling errors sharply along sensitive directions. At fixed information, reducing pixel error admits two dual geometric interventions: regularizing the decoder to flatten its sensitivity to latent variations, and aligning diffusion coordinates with the frozen decoder's metric at identical clean reconstruction. On ImageNet-256, aligning diffusion coordinates largely closes the guided generation penalty of multi-layer fusion and reaches state-of-the-art 0.984 gFID and 2.14 FDr6 in 80 epochs. RAEs should take information from the encoder and geometry from the decoder.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v3 · 2026-11-17 19:52 · 最新观测版本
Rethinking the Reconstructio-Generation Gap in Representation Autoencoders
Fusing shallower encoder layers into a Representation Autoencoder (RAE) sharpens pixel reconstruction but degrades diffusion generation. Prior studies and the research community commonly attribute this trade-off to the latent content itself, assuming detail-rich features are harder to model. We show that this gap stems from geometric mismatch at the encoder-decoder interface. Freezing a foundation encoder preserves rich visual information, but imposes an encoder-native Euclidean metric shaped without pixel decoding, whereas the decoder amplifies sampling errors sharply along sensitive directions. At fixed information, reducing pixel error admits two dual geometric interventions: regularizing the decoder to flatten its sensitivity to latent variations, and aligning diffusion coordinates with the frozen decoder's metric at identical clean reconstruction. On ImageNet-256, aligning diffusion coordinates largely closes the guided generation penalty of multi-layer fusion and reaches state-of-the-art 0.984 gFID and 2.14 FDr6 in 80 epochs. RAEs should take information from the encoder and geometry from the decoder.
查看此版本解读 →查看此版本原文 ↗