arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2610.19129cs.LGcs.AI
收藏

基于几何视界模型构想目标以实现策略泛化

Imagining Goals with Geometric Horizon Models for Policy Generalization

Mila;Université de Montréal(蒙特利尔大学);Google DeepMind(谷歌DeepMind)

Daniel Lawson, Dhruv Sreenivas, Adriana Hugessen, Khimya Khetarpal, Glen Berseth

AI总结:

本研究针对零样本目标泛化难题,提出TD-Aug方法,通过几何视界模型采样构想组合现有知识的新结果,将其作为目标训练目标条件化BC策略,显著提升了基于拼接的OGBench任务的泛化能力。

摘要:

从离线数据训练已在机器人学等领域取得显著进展,催生了可轻松零样本应用或高效微调至下游任务的通用策略。然而,这类策略仍存在泛化能力不足的问题,其原因既包括建模目标的选择,也源于从静态数据集学习。本研究聚焦于极具挑战性的零样本目标泛化任务,该任务需在未见任务上对策略进行评估,要求策略结合现有知识(即组合泛化)。提升策略泛化能力的一个途径是通过世界模型生成新经验,但此类生成在长视界情况下已被证实十分困难。因此,为缓解该问题,我们提出TD-Aug,该方法从几何视界模型中采样,可直接构想通过组合现有知识能实现的新结果。我们证明,将这些未来结果作为目标用于目标条件化BC策略的训练,可显著提升基于拼接的OGBench任务中的泛化能力。

英文摘要:

Training from offline data has allowed for substantial progress in domains such as robotics, leading to general-purpose policies that can be easily applied zero-shot or efficiently finetuned for downstream tasks. However, these policies can still have poor generalization, both due to the choice of modeling objective and from learning from a static dataset. In this work, we focus on the challenging task of zero-shot goal generalization, where a policy is evaluated on unseen tasks that require combining its existing knowledge (compositional generalization). An avenue for improving a policy's generalization is by generating new experience with world models; however, such generation has proven difficult for longer horizons. Thus, to alleviate this issue, we propose TD-Aug, which samples from a geometric horizon model and allows for directly imagining novel outcomes that can be achieved by composing existing knowledge. We demonstrate that training on these future outcomes as goals for goal-conditioned BC policies significantly improves generalization in stitching-based OGBench tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑