扩散语言模型并行解码中的实测耦合
Measured Coupling for Parallel Decoding in Diffusion Language Models
Tal Oved, Lior Wolf
AI 速读 基于摘要
用主前向传播内的廉价探针直接测量提交一个 token 对另一位置置信度的影响（耦合），替代均值场解码的相似度代理，以减少并行解码步数。
cs.CL · cs.AI · cs.DC · cs.LG · cs.PF
查看原文摘要
Diffusion language models can write many tokens per step, but must choose which tokens are safe to commit together. Mean-field decoding penalizes pairs of positions whose predictions look alike. We replace this proxy with the coupling itself: how much committing one token changes the model's confidence at another. Cheap probes inside the main forward pass measure it, with no training or extra model, and it plugs into the same mean-field rule. On three models and four benchmarks, our method needs far fewer steps than mean-field decoding and is faster in most settings, also with a key-value cache, at an average accuracy cost of 1.1 points. Shuffling the couplings or replacing them with a uniform bonus loses this gain.