← 研踪 PaperTrailarXiv:2605.23762 · v2
Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos
首次提交 2026-05-22 23:33
版本更新 2026-11-17 18:22
雷达观测 2026-11-18 11:33
摘要
Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. So far, dynamic retargeting methods obtain physically feasible references by tracking an intermediate geometric retargeting in simulation, usually through its joint angles. We show that this intermediate step biases the solution through two factors: the space in which the reference is tracked, joint angles rather than task-space keypoints, and the tracked reference itself, the keypoints of the geometric retargeting rather than those of the human demonstration. We propose Direct Dynamic Retargeting (DDR), which bypasses both: it tracks the human keypoints exclusively in task space with a sampling-based Model Predictive Control solver inside a physics simulator, without contact annotations. DDR is evaluated in simulation against joint-space dynamic retargeting under controlled conditions (identical solver, budget, and warm start), on three humanoids and on motions from monocular videos and motion capture. The tracking space explains most of the gain in end-effector accuracy and robustness, and tracking the human demonstration directly prevents the transmission of errors from the intermediate geometric reference. This advantage carries over to Reinforcement Learning imitation policies, which reproduce the human demonstration more faithfully than policies trained on joint-space dynamic references. These policies are deployed zero-shot on two physical humanoids of very different sizes, H1-2 and R1, including contact-rich motions. We release the source code and a dataset of dynamically feasible references covering all the flat-ground motions of AMASS on the three humanoids.
作者备注:https://gepetto.github.io/robot-motion/ddr/
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-17 18:22 · 最新观测版本
Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos
Imitation Learning from monocular video demonstrations provides a scalable approach for teaching complex skills to humanoid robots. However, translating human motion to humanoids requires overcoming significant morphological mismatches. So far, dynamic retargeting methods obtain physically feasible references by tracking an intermediate geometric retargeting in simulation, usually through its joint angles. We show that this intermediate step biases the solution through two factors: the space in which the reference is tracked, joint angles rather than task-space keypoints, and the tracked reference itself, the keypoints of the geometric retargeting rather than those of the human demonstration. We propose Direct Dynamic Retargeting (DDR), which bypasses both: it tracks the human keypoints exclusively in task space with a sampling-based Model Predictive Control solver inside a physics simulator, without contact annotations. DDR is evaluated in simulation against joint-space dynamic retargeting under controlled conditions (identical solver, budget, and warm start), on three humanoids and on motions from monocular videos and motion capture. The tracking space explains most of the gain in end-effector accuracy and robustness, and tracking the human demonstration directly prevents the transmission of errors from the intermediate geometric reference. This advantage carries over to Reinforcement Learning imitation policies, which reproduce the human demonstration more faithfully than policies trained on joint-space dynamic references. These policies are deployed zero-shot on two physical humanoids of very different sizes, H1-2 and R1, including contact-rich motions. We release the source code and a dataset of dynamically feasible references covering all the flat-ground motions of AMASS on the three humanoids.
查看此版本解读 →查看此版本原文 ↗