Reimagine Video Dynamics: A Unified Framework for Video Editing with Editable Spatiotemporal Tokens
Yu Yuan, Yawen Lu, Guoxian Song, Kevin Duarte, Ratheesh Kalarot, Di Chang, Xijun Wang, Stanley H. Chan
原文摘要
Most video editing methods can change how a scene looks but offer limited control over how it moves. We introduce Reimagine Video Dynamics (RVD), a unified framework for editing actions, motion pace, and appearance with reusable dynamics tokens and a shared renderer. The tokens capture motion from the source video and retain an explicit temporal axis. A language-guided editor transforms them to change an action or event. Resampling them changes the pace of motion, while keeping them fixed allows the renderer to reproduce the motion with a new appearance. We train the renderer to reconstruct source videos. We pretrain the editor on motion from real videos, then fine-tune it on counterfactual pairs. Across 450 dynamics edits, RVD achieves the highest instruction-following score and an 80.0\% pass rate. The same renderer supports retiming and motion reuse without additional task-specific training.
cs.CV