← 研踪 PaperTrailarXiv:2509.19573 · v2
Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning
首次提交 2025-09-24 04:57
版本更新 2026-11-10 13:01
雷达观测 2026-11-11 13:08
摘要
Achieving highly dynamic behaviors on humanoid robots, such as running, requires controllers that are both robust and precise, and hence difficult to design. Classical control theory offers principled tools for stabilizing such systems but struggles to synthesize real-time controllers for their nonlinear, hybrid dynamics; reinforcement learning (RL) handles these dynamics well but typically depends on extensive hand-tuning of heuristic rewards. In this work, we embed ideas from nonlinear control theory, specifically control Lyapunov functions (CLFs), along with optimized reference trajectories for the hybrid system into the reinforcement learning reward. This approach, CLF-RL, replaces the need to handcraft and tune heuristic reward terms with easy to tune stability informed rewards. We further develop a trajectory optimization routine that can handle the hybrid structure of the system to produce dynamically feasible, multi-domain, trackable references; using full-order dynamics rather than a kinematic approximation yields up to a 1.7x improvement in position tracking. The resulting policy runs on a Unitree G1 humanoid with a full flight phase, operating reliably in tight corridors such as a treadmill as well as in outdoor environments, all while remaining robust to disturbances applied to the torso and feet. Moreover, it achieves accurate global reference tracking utilizing only on-board sensors, making a critical step toward integrating these dynamic motions into a full autonomy stack.
作者备注：Accepted to Humanoids 2026
观测到的版本
相邻记录可能跳过中间版本；首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-10 13:01 · 最新观测版本
Chasing Stability: Humanoid Running via Control Lyapunov Function Guided Reinforcement Learning
Achieving highly dynamic behaviors on humanoid robots, such as running, requires controllers that are both robust and precise, and hence difficult to design. Classical control theory offers principled tools for stabilizing such systems but struggles to synthesize real-time controllers for their nonlinear, hybrid dynamics; reinforcement learning (RL) handles these dynamics well but typically depends on extensive hand-tuning of heuristic rewards. In this work, we embed ideas from nonlinear control theory, specifically control Lyapunov functions (CLFs), along with optimized reference trajectories for the hybrid system into the reinforcement learning reward. This approach, CLF-RL, replaces the need to handcraft and tune heuristic reward terms with easy to tune stability informed rewards. We further develop a trajectory optimization routine that can handle the hybrid structure of the system to produce dynamically feasible, multi-domain, trackable references; using full-order dynamics rather than a kinematic approximation yields up to a 1.7x improvement in position tracking. The resulting policy runs on a Unitree G1 humanoid with a full flight phase, operating reliably in tight corridors such as a treadmill as well as in outdoor environments, all while remaining robust to disturbances applied to the torso and feet. Moreover, it achieves accurate global reference tracking utilizing only on-board sensors, making a critical step toward integrating these dynamic motions into a full autonomy stack.
查看此版本解读 →查看此版本原文 ↗