← 研踪 PaperTrailCVF · CVPR 2026:cvf:9f628a14280ef320edb0fd330d4db993 · 记录 #1
DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO
来源日期（仅到年份） 2026
来源记录日期 2026
雷达观测 2026-11-01 23:24
摘要
Reinforcement learning (RL), particularly GRPO, improves image generation quality significantly by comparing the relative performance of images generated within the same group. However, in the later stages of training, the model tends to produce homogenized outputs, lacking creativity and visual diversity, restricting the application scenarios of the model. This issue can be analyzed from both reward modeling and generation dynamics perspectives. First, traditional GRPO relies on single sample quality as the reward signal, driving the model to converge toward a few high reward generation modes while neglecting distribution-level diversity. Second, conventional GRPO regularization neglects the dominant role of early stage denoising in preserving diversity, causing a misaligned regularization budget that limits the achievable quality-diversity trade-off.Motivated by these insights, we revisit the diversity degradation problem from both reward modeling and generation dynamics. At the reward level, we propose a distributional creativity bonus based on semantic grouping. Specifically, we construct a distribution-level representation via spectral clustering over samples generated from the same caption, and adaptively allocate exploratory rewards according to group sizes to encourage the discovery of novel visual modes. At the generation level, we introduce a structure-aware regularization, which enforces stronger early-stage constraints to preserve diversity without compromising reward optimization efficiency. Experiments demonstrate that our method achieves an 13%~18% improvement in semantic diversity under matched quality scores, establishing a new Pareto frontier between image quality and diversity for GRPO-based image generation.
论文发现出处
这些页面提供论文线索，不代表同行评审或质量认证。
观测到的版本
平台记录号代表本站观测到的元数据变化，不是作者修订号。来源日期仅到月份或年份时，按该月或该年首日存档。
记录 #1 · 2026 · 最新观测版本
DiverseGRPO: Mitigating Mode Collapse in Image Generation via Diversity-Aware GRPO
Reinforcement learning (RL), particularly GRPO, improves image generation quality significantly by comparing the relative performance of images generated within the same group. However, in the later stages of training, the model tends to produce homogenized outputs, lacking creativity and visual diversity, restricting the application scenarios of the model. This issue can be analyzed from both reward modeling and generation dynamics perspectives. First, traditional GRPO relies on single sample quality as the reward signal, driving the model to converge toward a few high reward generation modes while neglecting distribution-level diversity. Second, conventional GRPO regularization neglects the dominant role of early stage denoising in preserving diversity, causing a misaligned regularization budget that limits the achievable quality-diversity trade-off.Motivated by these insights, we revisit the diversity degradation problem from both reward modeling and generation dynamics. At the reward level, we propose a distributional creativity bonus based on semantic grouping. Specifically, we construct a distribution-level representation via spectral clustering over samples generated from the same caption, and adaptively allocate exploratory rewards according to group sizes to encourage the discovery of novel visual modes. At the generation level, we introduce a structure-aware regularization, which enforces stronger early-stage constraints to preserve diversity without compromising reward optimization efficiency. Experiments demonstrate that our method achieves an 13%~18% improvement in semantic diversity under matched quality scores, establishing a new Pareto frontier between image quality and diversity for GRPO-based image generation.
查看此版本解读 →查看此版本原文 ↗