EgoGeneralist：通过视频原生建模统一第一人称动作-视频任务
EgoGeneralist: Unifying Egocentric Action-Video Tasks through Video-Native Modeling
Yanlin Jin, Bowen Yang, Cheng Zhang, Yifan Yang, Zhe Li, Jianfei Cai, Kai Zhu
AI 速读 基于摘要
作者提出 EgoGeneralist，将预训练视频生成器适配为统一处理动作条件第一人称视频生成、姿态估计与联合动作-视频生成的通用框架。
cs.CV
查看原文摘要
Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts pretrained video generators to unify action-conditioned egocentric video generation, pose estimation, and joint action-video generation. Our role-adaptive action renderer provides video-native representations for multidirectional action-video learning, making metric motion usable as both a spatial control signal and a recoverable generation target. Cross-stream action-video coupling establishes close correspondence between metric motion and visual dynamics through lightweight adaptation while preserving the pretrained visual prior. Together, these designs establish a generalist framework spanning five action-video tasks. Despite adaptation only on controlled laboratory recordings, EgoGeneralist generalizes to diverse unseen scenes and interactions with effective action control and visual quality, bridging pretrained visual knowledge and metric 3D human motion.