SpecCTMC：扩散语言模型连续时间马尔可夫链的精确推测模拟
SpecCTMC: Exact Speculative Simulation of Diffusion Language Model Continuous-Time Markov Chains
Siva Rajesh Kasa, Yasong Dai, Sumit Negi, Hongdong Li
AI 速览：提出一种无需草稿模型的免训练推测采样器，在保持连续时间马尔可夫链联合分布精确不变的前提下，减少扩散语言模型顺序模型评估次数。
Masked diffusion language models offer parallel text generation, but independently sampling multiple tokens can disrupt their dependencies and change the output distribution. Exact continuous-time sampling avoids this approximation, yet typically requires one sequential model evaluation per token. We introduce SpecCTMC, a training-free speculative sampler that reduces this sequential bottleneck without a separate draft model. SpecCTMC proposes successive token updates using previously computed model predictions, evaluates their intermediate states in parallel, and applies an exact acceptance-and-correction procedure. We prove that SpecCTMC preserves the joint distribution of event times, generation order, and token values of the underlying continuous-time Markov chain. It therefore introduces neither time-discretization error nor additional factorization error from parallel decoding. Experiments on LLaDA-8B-Instruct and Dream-v0-Instruct-7B across five benchmarks demonstrate 1.73-4.38x fewer sequential model evaluations than direct continuous-time sampling, while preserving the reference sampling distribution. Our code is available at: https://github.com/kasakh/specctmc