← 研踪 PaperTrailarXiv:2611.13120 · v1
Just Image Transformers with Late-Stage Pixel Refinement
首次提交 2026-11-10 03:08
版本更新 2026-11-10 03:08
雷达观测 2026-11-11 11:04
摘要
Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image (-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of (from with corrected sampling), without using additional losses.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-10 03:08 · 最新观测版本
Just Image Transformers with Late-Stage Pixel Refinement
Recent work on latent-free diffusion transformers has demonstrated the feasibility of training image generators without a latent VAE. This avoids compression artifacts from the VAE and simplifies the training pipeline. In particular, JiT showed that even a standard transformer can be trained to generate images when the model is trained to predict the final image (-prediction). Even though JiT unlocks training efficiency comparable to latent models, it still suffers from reduced image quality likely arising from modeling the image as a collection of large patches. In this work, we revisit JiT and show that adding simple convolutional pixel-pathway layers in a post-training phase significantly improves generation quality. Moreover, we show that the added pixel-pathway does not need to be active in the early denoising stages, and can be activated as late as in the last of the denoising process. Our experiments on ImageNet@256 show improvements compared to the original JiT results, achieving an FID of (from with corrected sampling), without using additional losses.
查看此版本解读 →查看此版本原文 ↗