注意差距:LeWorldModel中分层规划的前景与陷阱
Mind the Gap: Promises and Pitfalls of Hierarchical Planning in LeWorldModel
- University of Amsterdam(阿姆斯特丹大学)
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
研究长期目标条件控制下时间层次结构对LeWorldModel的影响,提出Hi-LeWM扩展,通过冻结低级模型并添加高级规划。实验发现层次结构需合理设计,无约束搜索有问题,约束搜索可改善性能,表明时间抽象有益,但高级搜索要与低级控制器兼容。
AI中文摘要:
我们研究时间层次结构能否在长期目标条件控制中提升LeWorldModel。我们引入了Hi-LeWM,它冻结预训练的低级LeWorldModel并在潜在子目标上添加高级规划。我们在PushT和Cube上评估Hi-LeWM,随着目标偏移增加。层次结构不会自动提升性能,短时间内最佳配置使用单步高级时间范围,长时间则显示出学习到的高级动作空间与推理时搜索分布不匹配。实验表明冻结的低级控制器能执行良好对齐的中间目标,高级子目标生成是主要瓶颈。无约束搜索会选择在学习模型下看似有利但产生不良控制目标的潜在宏动作。通过在训练轨迹编码的宏动作周围约束搜索,并配合适当的子目标执行时间,可恢复有用的层次结构模式,在中程范围比平面LeWorldModel提高11.3个百分点,在最长PushT范围提高14.7个百分点。总体而言,时间抽象可使紧凑的冻结LeWorldModel受益,但前提是高级搜索与低级控制器兼容。
英文摘要:
We investigate whether temporal hierarchy can improve LeWorldModel on long-horizon goal-conditioned control. We introduce Hi-LeWM, an extension that freezes the pretrained low-level LeWM and adds high-level planning over latent subgoals. We evaluate Hi-LeWM on PushT and Cube across increasing goal offsets. Hierarchy does not automatically improve performance: at short horizons, the best configuration uses a one-step high-level horizon, while longer horizons reveal a mismatch between the learned high-level action space and the inference-time search distribution. Experiments with true future latent subgoals show that the frozen low-level controller can execute well-aligned intermediate targets, indicating that high-level subgoal generation is the main bottleneck. Unconstrained search can select latent macro-actions that appear favorable under the learned model but produce poor control targets. Constraining search around macro-actions encoded from training trajectories, with appropriate subgoal execution timing, recovers useful hierarchical regimes, improving over flat LeWM by +11.3 percentage points at medium-range horizons and +14.7 percentage points at the longest PushT horizon. Overall, temporal abstraction can benefit compact frozen LeWM, but only when high-level search remains compatible with the low-level controller