Steering Away from Memorization: Reachability-Constrained Reinforcement Learning for Text-to-Image Diffusion
Sathwik Karnik, Juyeop Kim, Sanmi Koyejo, Jong-Seok Lee, Somil Bansal
原文摘要
Text-to-image diffusion models are susceptible to memorization, revealing a fundamental failure to generalize beyond the training set. Current mitigation approaches often sacrifice image quality or prompt alignment to reduce memorization. To address this, we propose Reachability-Aware Diffusion Steering (RADS), an offline-trained, sampling-time control framework that mitigates memorization while maintaining generation fidelity. RADS models the diffusion denoising process as a dynamical system and learns a policy-conditioned reachability value function that estimates future memorization risk along the denoising trajectory. This motivates a constrained reinforcement learning (RL) formulation, where a policy learns to steer the trajectory away from memorization via perturbations in the caption embedding space. Empirical evaluations show that RADS achieves a stronger empirical Pareto frontier between generation diversity (SSCD), quality (FID), and alignment (CLIP) compared to state-of-the-art inference-time baselines. Crucially, RADS applies the learned controller during sampling time, separating memorization mitigation from the pretrained model. Our website is available at: https://s-karnik.github.io/rads-memorization-project-page/.
cs.CV · cs.AI · cs.LG