← 研踪 PaperTrailarXiv:2611.20791 · v1
QTOP: Scaling Humanoid Motion Tracking with Quantile Critics
首次提交 2026-11-16 17:37
版本更新 2026-11-16 17:37
雷达观测 2026-11-17 17:11
摘要
Learning human motion tracking policies through reinforcement learning (RL) has emerged as a core capability that underpins a majority of recent progress in humanoid whole-body control. However, achieving generalization to unseen motions at high fidelity requires jointly scaling training to thousands of GPU and dataset hours. This work hypothesizes that motion tracking is inherently compute-bound because training on a diverse set of motions introduces learning interference, similar to multi-task RL. To tackle this, we propose Quantile Tracking Optimization (QTOP), which improves sample efficiency by introducing a distributional advantage estimate conditioned on state and reference pairs. QTOP outperforms on and off-policy RL baselines both in simulation and real-world tracking scenarios on multiple datasets by up to 10\%. Furthermore, we establish a scaling law for QTOP and find that, aside from scaling predictably, dataset size has minimal impact on performance for moderately sized GPU hour regimes.
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v1 · 2026-11-16 17:37 · 最新观测版本
QTOP: Scaling Humanoid Motion Tracking with Quantile Critics
Learning human motion tracking policies through reinforcement learning (RL) has emerged as a core capability that underpins a majority of recent progress in humanoid whole-body control. However, achieving generalization to unseen motions at high fidelity requires jointly scaling training to thousands of GPU and dataset hours. This work hypothesizes that motion tracking is inherently compute-bound because training on a diverse set of motions introduces learning interference, similar to multi-task RL. To tackle this, we propose Quantile Tracking Optimization (QTOP), which improves sample efficiency by introducing a distributional advantage estimate conditioned on state and reference pairs. QTOP outperforms on and off-policy RL baselines both in simulation and real-world tracking scenarios on multiple datasets by up to 10\%. Furthermore, we establish a scaling law for QTOP and find that, aside from scaling predictably, dataset size has minimal impact on performance for moderately sized GPU hour regimes.
查看此版本解读 →查看此版本原文 ↗