跳到正文
研踪PaperTrail
← 研踪 PaperTrail

arXiv:2610.14335 · v2

PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Imagined Subgoals and Policy Priors

首次提交 2026-10-10 05:21
版本更新 2026-11-19 11:47
雷达观测 2026-11-20 10:50
arXiv ↗PDF ↗

摘要

Latent world-model planning has shown promising results in goal-conditioned robotic manipulation, but remains challenging for long-horizon planning due to limited temporal guidance and inefficient action search. A distant goal image offers little guidance for preparatory actions, capturing intermediate subgoal images by hand is impractical, and uninformed sampling makes action search inefficient. We propose PILOT (Policy-Initialized Local Optimization with Temporal subgoals), which brings two foundation models into latent world-model planning: a video model turns a language instruction into visual subgoals, and a vision-language-action policy initializes the cross-entropy method (CEM). CEM refines action sequences toward each subgoal with an action-conditioned world model, and no goal image is needed at test time. With both models fine-tuned on 40 demonstrations, PILOT achieves 68.8% success on four real UR5e tasks, compared with 55.0% without policy initialization, 13.8% for policy-only execution, and 0.0% for direct final-goal planning. In simulation, PILOT exceeds direct execution of the same policy by 30 percentage points on long Push-T tasks and by 22 percentage points on LIBERO-10. Subgoals and the policy prior are complementary: better subgoals and stronger priors each raise success, and even a weak policy that succeeds alone on only 3.3% of medium Push-T tasks is an effective prior.

AI 论文解读

基于摘要 · 非全文审阅

此版本尚未生成 AI 解读。解读由后台任务生成并按论文版本保存,登录后共享已有结果。

版本差异

本站首次收录为 v2,暂无更早版本可对照。

观测到的版本

相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。

v2 · 2026-11-19 11:47 · 最新观测版本

PILOT: Guiding Long-Horizon World-Model Planning for Robot Manipulation with Imagined Subgoals and Policy Priors

Latent world-model planning has shown promising results in goal-conditioned robotic manipulation, but remains challenging for long-horizon planning due to limited temporal guidance and inefficient action search. A distant goal image offers little guidance for preparatory actions, capturing intermediate subgoal images by hand is impractical, and uninformed sampling makes action search inefficient. We propose PILOT (Policy-Initialized Local Optimization with Temporal subgoals), which brings two foundation models into latent world-model planning: a video model turns a language instruction into visual subgoals, and a vision-language-action policy initializes the cross-entropy method (CEM). CEM refines action sequences toward each subgoal with an action-conditioned world model, and no goal image is needed at test time. With both models fine-tuned on 40 demonstrations, PILOT achieves 68.8% success on four real UR5e tasks, compared with 55.0% without policy initialization, 13.8% for policy-only execution, and 0.0% for direct final-goal planning. In simulation, PILOT exceeds direct execution of the same policy by 30 percentage points on long Push-T tasks and by 22 percentage points on LIBERO-10. Subgoals and the policy prior are complementary: better subgoals and stronger priors each raise success, and even a weak policy that succeeds alone on only 3.3% of medium Push-T tasks is an effective prior.

查看此版本解读 →查看此版本原文 ↗