Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation
Minseok Joo, Dogyun Park, Taehoon Lee, Kyujin Lee, Hyunwoo J. Kim
原文摘要
Long-horizon video generation is a key capability of video world models, yet maintaining geometric consistency remains challenging. Memory-augmented generative models address this by retrieving historical frames, but their effectiveness depends on two key design choices: what 3D-geometric evidence should represent past observations, and how memory frames should be selected from this evidence. Pose-based field-of-view overlap provides lightweight but coarse geometric evidence, while explicit 3D scene memories offer finer spatial information at the cost of geometry estimation, alignment, and incremental updates over long rollouts. We propose Coverage-Maximizing Retrieval-Augmented Generation (CovRAG), a depth-based memory retrieval framework that uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D memory evidence. For frame selection, CovRAG maximizes residual coverage gain, iteratively retrieving frames that explain target-view regions not covered by the temporal context or previously selected memories. To improve scalability in long-video generation, we introduce sliding-window depth caching for efficient geometry estimation. Experiments on RealEstate10K and DL3DV10K show that CovRAG improves long-horizon geometric consistency over the evaluated memory-retrieval baselines while maintaining low retrieval latency.
cs.CV