跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2611.26586 · v1

ProtoDiffusion: Token-per-Layer Generative Modeling of Multi-Layer Images

ProtoDiffusion:面向多层图像的逐层令牌生成建模

首次提交 2026-11-20 02:45
版本更新 2026-11-20 02:45
雷达观测 2026-11-20 12:52
arXiv ↗PDF ↗

摘要

Graphic designs are ordered compositions of RGBA layers whose separation enables selection, movement, and reuse. Existing layered models represent each layer with a full-canvas or region-sized spatial latent grid, so representation size grows with the cumulative spatial extent of the layers. We present ProtoDiffusion, a latent-diffusion framework built on ProtoVAE, a multi-layer variational autoencoder that encodes a design into one shared spatial prototype and one candidate vector per layer. Each candidate conditions a shared decoder to reconstruct its corresponding RGBA layer. For NN layers, the representation contains HW+NHW+N tokens before backbone-specific packing, compared with NHWNHW for full-grid-per-layer representations. ProtoDiffusion jointly denoises the prototype and a fixed set of candidate slots, conditioned on an image or text, with bounding boxes as optional layout guidance. The model leaves unneeded slots empty and determines the number of layers and, in absence of boxes, their layout, requiring neither as input. The same architecture supports both image-to-design and text-to-design. ProtoVAE achieves high-fidelity reconstruction from its compact shared latent representation, reaching 40.43 dB PSNR on layer crops. On the LayerD evaluation split, image-to-design achieves RGB L1 scores of 0.040 with bounding boxes and 0.057 without, with box-guided alpha soft IoU (0.850) comparable to the strongest fine-tuned baseline (0.871). On merged images, text-to-design reaches 66.41 FID versus 86.84 for FLUX.2 [klein], achieves a 0.860 text-accuracy score, and runs 19.1×19.1\times faster than the same-backbone generation-then-decomposition cascade.

作者备注:20 pages, 16 figures

AI 论文解读

基于摘要 · 非全文审阅

ProtoDiffusion 提出一种基于共享空间原型和每层候选向量的潜在扩散框架,以 HW+N 个令牌表示 N 层 RGBA 设计,支持图像到设计和文本到设计。

正在读取账号与研究记录…

版本差异

目前只收录了一个版本。收录下一版后,可以对照标题、作者、分类和摘要。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v1 · 2026-11-20 02:45 · 最新观测版本

ProtoDiffusion: Token-per-Layer Generative Modeling of Multi-Layer Images

Graphic designs are ordered compositions of RGBA layers whose separation enables selection, movement, and reuse. Existing layered models represent each layer with a full-canvas or region-sized spatial latent grid, so representation size grows with the cumulative spatial extent of the layers. We present ProtoDiffusion, a latent-diffusion framework built on ProtoVAE, a multi-layer variational autoencoder that encodes a design into one shared spatial prototype and one candidate vector per layer. Each candidate conditions a shared decoder to reconstruct its corresponding RGBA layer. For NN layers, the representation contains HW+NHW+N tokens before backbone-specific packing, compared with NHWNHW for full-grid-per-layer representations. ProtoDiffusion jointly denoises the prototype and a fixed set of candidate slots, conditioned on an image or text, with bounding boxes as optional layout guidance. The model leaves unneeded slots empty and determines the number of layers and, in absence of boxes, their layout, requiring neither as input. The same architecture supports both image-to-design and text-to-design. ProtoVAE achieves high-fidelity reconstruction from its compact shared latent representation, reaching 40.43 dB PSNR on layer crops. On the LayerD evaluation split, image-to-design achieves RGB L1 scores of 0.040 with bounding boxes and 0.057 without, with box-guided alpha soft IoU (0.850) comparable to the strongest fine-tuned baseline (0.871). On merged images, text-to-design reaches 66.41 FID versus 86.84 for FLUX.2 [klein], achieves a 0.860 text-accuracy score, and runs 19.1×19.1\times faster than the same-backbone generation-then-decomposition cascade.

查看此版本解读 →查看此版本原文 ↗