离散扩散模型的改进采样调度
Improved Sampling Schedules for Discrete Diffusion Models
Alberto Foresti, Mustapha Bounoua, Giulio Franzese, Luca Ambrogioni, Pietro Michiardi
AI 速览：作者从热力学熵产生与动力学活动角度分析均匀离散扩散模型的反向过程，并提出两种新的采样调度（EDS、WDS），在语言建模任务上报告了计算效率提升。
Uniform Discrete Diffusion Models have emerged as a powerful paradigm for generative modeling on sequence data; however, the information-theoretic principles governing their reverse processes remain underexplored. In this work, we bridge this gap by analyzing the reverse process dynamics through the lens of thermodynamic entropy production and dynamical activity. We use a variational expression of the discrete Wasserstein distance to characterize the dynamics of the generation process. Leveraging these insights, we first introduce the Entropic Discrete Schedule (EDS), which aligns with entropy production, and later construct the Wasserstein Discrete Schedule (WDS), which combines entropy production and dynamical activity to act as a geometric cost. Our experimental results on language modeling show that WDS provide the same results as uniform while using up to half of the computations on the MBPP benchmark. The code is available at https://github.com/AlbertoForesti/impscheds.