← 研踪 PaperTrailarXiv:2606.02479 · v2
Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation
首次提交 2026-06-02 00:49
版本更新 2026-11-07 20:48
雷达观测 2026-11-10 11:14
摘要
Long-horizon video generation is a key capability of video world models, yet maintaining geometric consistency remains challenging. Memory-augmented generative models address this by retrieving historical frames, but their effectiveness depends on two key design choices: what 3D-geometric evidence should represent past observations, and how memory frames should be selected from this evidence. Pose-based field-of-view overlap provides lightweight but coarse geometric evidence, while explicit 3D scene memories offer finer spatial information at the cost of geometry estimation, alignment, and incremental updates over long rollouts. We propose Coverage-Maximizing Retrieval-Augmented Generation (CovRAG), a depth-based memory retrieval framework that uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D memory evidence. For frame selection, CovRAG maximizes residual coverage gain, iteratively retrieving frames that explain target-view regions not covered by the temporal context or previously selected memories. To improve scalability in long-video generation, we introduce sliding-window depth caching for efficient geometry estimation. Experiments on RealEstate10K and DL3DV10K show that CovRAG improves long-horizon geometric consistency over the evaluated memory-retrieval baselines while maintaining low retrieval latency.
作者备注：Accepted at NeurIPS 2026. Project page: https://mlvlab.github.io/CovRAG
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-07 20:48 · 最新观测版本
Retrieve What's Missing: Coverage-Maximizing Retrieval for Consistent Long Video Generation
Long-horizon video generation is a key capability of video world models, yet maintaining geometric consistency remains challenging. Memory-augmented generative models address this by retrieving historical frames, but their effectiveness depends on two key design choices: what 3D-geometric evidence should represent past observations, and how memory frames should be selected from this evidence. Pose-based field-of-view overlap provides lightweight but coarse geometric evidence, while explicit 3D scene memories offer finer spatial information at the cost of geometry estimation, alignment, and incremental updates over long rollouts. We propose Coverage-Maximizing Retrieval-Augmented Generation (CovRAG), a depth-based memory retrieval framework that uses pretrained 3D priors to construct a target-view coverage map as lightweight 3D memory evidence. For frame selection, CovRAG maximizes residual coverage gain, iteratively retrieving frames that explain target-view regions not covered by the temporal context or previously selected memories. To improve scalability in long-video generation, we introduce sliding-window depth caching for efficient geometry estimation. Experiments on RealEstate10K and DL3DV10K show that CovRAG improves long-horizon geometric consistency over the evaluated memory-retrieval baselines while maintaining low retrieval latency.
查看此版本解读 →查看此版本原文 ↗