← 研踪 PaperTrailarXiv:2610.00921 · v2
World-Model Gradient Refinement in CEM: An Empirical Study
首次提交 2026-10-01 09:57
版本更新 2026-11-14 19:46
雷达观测 2026-11-17 12:06
摘要
The cross-entropy method (CEM) uses world-model predictions both to select action sequences and to determine where to sample next. We isolate the effect of an early update by replacing only the first model-ranked elite set with environment-ranked elites. This fixed-state intervention lowers the selected sequence's realised tracking cost in all 12 Walker/Cheetah task-seed units. We then study short gradient updates to top-ranked candidates as a way to improve search under the model's own objective. Across 52 frozen TD-MPC2 checkpoints from 32 tasks, we compare refinement with additional CEM sampling at the same planning budget, tuning both planners' initial proposal widths. We fix the groups before this comparison using an exploratory model-only diagnostic. Refinement gains 67.8 return points on average (95\% CI [31.9, 103.6]) on ten checkpoints with large model-predicted improvements, and 0.2 ([, 3.4]) on the remaining 42. The diagnostic's rank test on 24 new checkpoints fails, so the association does not establish a predictive rule. Offline analyses show local benefits on some tasks and positive estimates for both proposal refitting and final selection. End-of-search polishing also helps, leaving the need for repeated refitting unresolved.
作者备注:10 pages of main text, 2 pages of references, and 6 pages of appendices
观测到的版本
相邻记录可能跳过中间版本;首次发现 v2 及以上也会标为版本更新。
v2 · 2026-11-14 19:46 · 最新观测版本
World-Model Gradient Refinement in CEM: An Empirical Study
The cross-entropy method (CEM) uses world-model predictions both to select action sequences and to determine where to sample next. We isolate the effect of an early update by replacing only the first model-ranked elite set with environment-ranked elites. This fixed-state intervention lowers the selected sequence's realised tracking cost in all 12 Walker/Cheetah task-seed units. We then study short gradient updates to top-ranked candidates as a way to improve search under the model's own objective. Across 52 frozen TD-MPC2 checkpoints from 32 tasks, we compare refinement with additional CEM sampling at the same planning budget, tuning both planners' initial proposal widths. We fix the groups before this comparison using an exploratory model-only diagnostic. Refinement gains 67.8 return points on average (95\% CI [31.9, 103.6]) on ten checkpoints with large model-predicted improvements, and 0.2 ([, 3.4]) on the remaining 42. The diagnostic's rank test on 24 new checkpoints fails, so the association does not establish a predictive rule. Offline analyses show local benefits on some tasks and positive estimates for both proposal refitting and final selection. End-of-search polishing also helps, leaving the need for repeated refitting unresolved.
查看此版本解读 →查看此版本原文 ↗