面向移动相机的透视视频转360°视频的3D驱动框架
A 3D-Driven Framework for Moving-Camera Perspective-to-360° Video Generation
Michael Wakeham, Mrinmoy Kumar Das, Wenqi Wang, Ayush Tewari, Boqing Gong, Deepti Ghadiyaram
AI 速读 基于摘要
提出一种模块化、3D驱动的框架,将移动相机透视视频转360°视频的任务拆分为显式3D重建、全景图像补全与共享3D场景锚定三个组件。
cs.CV
查看原文摘要
Turning a moving-camera perspective video into a 360° video means generating content outside the input frame under several constraints: observations revealed over time must be preserved, unseen regions plausibly completed, and the resulting scene kept consistent as the camera viewpoint changes. Existing methods assign these different requirements to a single conditional video model, requiring them to be jointly learned from panoramic video data. We instead present a modular, 3D-driven framework that distributes these three responsibilities across specialized components, all connected through a shared scene representation. Input observations are reconstructed explicitly in 3D, while a pretrained panoramic image model completes unobserved content from a selected panoramic render. This generated content is lifted into the shared 3D scene, where it remains spatially anchored across camera viewpoints. Without requiring any additional panoramic-video training, our framework achieves state-of-the-art input-view fidelity, perceptual quality, and geometric consistency on a static-scene subset of the Argus evaluation set. These results support our decomposed, 3D-driven formulation as a promising alternative to direct video generation in this setting.