Trainspotting:从发现照片中学习以物体为中心的三维重建
Trainspotting: Learning Object-Centric 3D Reconstruction from Found Photos
Hanyu Chen, Noah Snavely
AI 速读 基于摘要
作者构建了来自 Wikimedia Commons 的、以物体为中心的多视角照片数据集 Trainspotting,并指出当前前馈重建模型在物体移动或多实体共享设计的场景下表现较差,而用该数据集微调 VGGT-Ω 可改善位姿、深度与几何重建。
cs.CV
查看原文摘要
State-of-the-art feedforward reconstruction models perform poorly on in-the-wild object-centric photo collections. They are trained largely on single-session object captures whose images vary in viewpoint while the object and its environment remain nearly fixed. Public photo archives contain a different form of multi-view data. They hold found photos of the same object or design: photos taken independently by many people across years, environments, cameras, and sometimes physical units. In this work, we introduce Trainspotting, a dataset of 12,068 objects and 311,886 photos from Wikimedia Commons. We reconstruct each object to recover camera poses, multi-view stereo depth, and object masks. We categorize the objects into three regimes: objects fixed at one site, objects that move between sites, and designs shared by many physical units. Evaluating recent feedforward reconstruction models on Trainspotting shows that they fail most severely in the last two regimes. Fine-tuning VGGT-$Ω$ on Trainspotting significantly improves camera pose, depth, and 3D geometry on held-out objects. We observe similar improvements on the independent NAVI in-the-wild benchmark, which contains photos of each object in different environments. Performance on conventional benchmarks remains largely unchanged.