← 研踪 PaperTrailarXiv:2606.30058 · v2
Emergence of a Shared Canonical Object Frame from In-the-Wild Videos
首次提交 2026-06-29 17:48
版本更新 2026-11-06 04:10
雷达观测 2026-11-09 10:16
摘要
Comparing object orientations and positions across different instances requires their poses to be expressed in a shared canonical frame. Establishing such frames has traditionally required manual annotation, creating a scaling bottleneck that limits category and instance diversity. We show that a shared canonical frame can instead emerge from self-supervised training on object-centric videos of rigid objects captured in the wild, using only noisy camera poses from Structure-from-Motion. Our key idea is to route all training sequences through a shared geometric bottleneck: a coarse canonical mesh that carries no category-specific detail. By learning dense correspondences from image pixels to this mesh, and estimating per-sequence alignments from noisy SfM geometry, a common canonical frame emerges from multi-view consistency and the semantic priors of the feature extractor, without any canonical pose labels or category conditioning. Trained in a self-supervised manner on ${\sim}160{,}000$ in-the-wild object videos, our method achieves competitive accuracy on category-level pose estimation benchmarks compared to methods that rely on canonical pose supervision. The code and checkpoint are available on \href{https://github.com/Fischer-Tom/Emergent-Canonical-Frame/}{github.com/Fischer-Tom/Emergent-Canonical-Frame/}.
作者备注：Accepted at NeurIPS-2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-06 04:10 · 最新观测版本
Emergence of a Shared Canonical Object Frame from In-the-Wild Videos
Comparing object orientations and positions across different instances requires their poses to be expressed in a shared canonical frame. Establishing such frames has traditionally required manual annotation, creating a scaling bottleneck that limits category and instance diversity. We show that a shared canonical frame can instead emerge from self-supervised training on object-centric videos of rigid objects captured in the wild, using only noisy camera poses from Structure-from-Motion. Our key idea is to route all training sequences through a shared geometric bottleneck: a coarse canonical mesh that carries no category-specific detail. By learning dense correspondences from image pixels to this mesh, and estimating per-sequence alignments from noisy SfM geometry, a common canonical frame emerges from multi-view consistency and the semantic priors of the feature extractor, without any canonical pose labels or category conditioning. Trained in a self-supervised manner on ${\sim}160{,}000$ in-the-wild object videos, our method achieves competitive accuracy on category-level pose estimation benchmarks compared to methods that rely on canonical pose supervision. The code and checkpoint are available on \href{https://github.com/Fischer-Tom/Emergent-Canonical-Frame/}{github.com/Fischer-Tom/Emergent-Canonical-Frame/}.
查看此版本解读 →查看此版本原文 ↗