基于显式对应关系的一致视频到视频翻译
Consistent Video-to-Video Translation via Explicit Correspondences
Gaurav Parmar, Zhengqi Li, Richard Zhang, Jun-Yan Zhu, Srinivasa Narasimhan, Eli Shechtman, Yotam Nitzan
AI 速读 基于摘要
作者提出 vid2vid-long，利用输入视频中的 patch 级对应关系检索并复用历史生成结果，以在保持实时性的同时提升长程时序一致性。
cs.CV · cs.AI · cs.GR · cs.LG
查看原文摘要
Interactive video-to-video applications require real-time generation while maintaining long-range temporal consistency. However, recent methods achieve speed by restricting attention to a fixed-length causal window. This limits long-range consistency whenever relevant history falls outside the context window. To address this limitation, we introduce vid2vid-long, a correspondence-based approach grounded in a simple observation. Video-to-video translation provides an explicit cross-frame reference: the regions that match in the input video should match in the output video too. Concretely, we compute patch-level correspondences in the input, guiding retrieval of previously generated outputs in the corresponding regions, and condition the model on these patches through correspondence cross-attention. Our experiments demonstrate that vid2vid-long consistently improves long-range temporal consistency across recent video models while preserving generation quality and real-time performance.