ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation
Baorui Peng, Wenyao Zhang, Liang Xu, Bohan Li, Hu Zhu, Zekun Qi, Hongsi Liu, Jiazhao Zhang 等 10 位作者
原文摘要
Current embodied video generation models primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. In this paper, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video generation with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based video generators using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.
cs.RO · cs.CV