Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos
Keru Zhou, Zhirui Fang, Xinrun Xu, Yaopeng Lou, Zeyuan He, Zihao Zhu, Xinze Niu, Chen Wang 等 22 位作者
原文摘要
Egocentric videos offer a low-cost, scalable source of manipulation experience for robot learning. However, existing methods often fail to tailor their use of egocentric videos and robot demonstrations to the differences in their formats and information content. We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. The global module learns task context through future video prediction, while the interaction module uses this context to interpret wrist observations and learns local execution information through robot action supervision. The distinct learning objectives enable independent pretraining on large-scale human and robot egocentric videos to learn task context, and training on real-robot demonstrations to align the learned representations with fine-grained robot actions. Experiments show leading performance on LIBERO-Plus and RoboTwin 2.0, while real-robot evaluations demonstrate generalization across tasks and embodiments.
cs.RO