arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.18759cs.ROcs.AIcs.LG
收藏

基于语言引导的状态抽象的学习方法

Learning with Language-Guided State Abstractions

MIT(麻省理工学院);Princeton(普林斯顿大学)

Andi Peng, Ilia Sucholutsky, Belinda Z. Li, Theodore R. Sumers, Thomas L. Griffiths, Jacob Andreas, Julie A. Shah

AI总结:

该研究提出LGA(语言引导抽象)框架,借助自然语言监督与预训练语言模型自动生成适配未见任务的状态抽象,用于模仿学习,可在短时间内达到接近人工设计的效果,提升策略的泛化性与鲁棒性,并在Spot机器人移动操作任务中验证了实用性。

摘要:

我们提出了一种利用自然语言为模仿学习设计状态抽象的框架。在高维观测空间中,精心设计的状态表征能够凸显环境的重要特征并屏蔽无关特征,从而助力可泛化策略的学习。这类状态表征通常需人工指定,或通过其他耗费人力的标注流程推导得到。我们的方法LGA(语言引导抽象)结合自然语言监督与语言模型(LM)的背景知识,可自动构建适配未见任务的状态表征。在LGA中,用户首先用自然语言提供目标任务的(可能不完整的)描述;随后,预训练LM将该任务描述转换为可屏蔽无关特征的状态抽象函数;最后,利用少量演示数据与LGA生成的抽象状态训练模仿策略。在模拟机器人任务上的实验表明,LGA生成的状态抽象与人工设计的效果相近,但耗时仅为后者的一小部分,且在存在伪相关和模糊规范的情况下,这些抽象能提升泛化性与鲁棒性。我们还在Spot机器人的移动操作任务上验证了所学抽象的实用性。

英文摘要:

We describe a framework for using natural language to design state abstractions for imitation learning. Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations, which can surface important features of an environment and hide irrelevant ones. These state representations are typically manually specified, or derived from other labor-intensive labeling procedures. Our method, LGA (language-guided abstraction), uses a combination of natural language supervision and background knowledge from language models (LMs) to automatically build state representations tailored to unseen tasks. In LGA, a user first provides a (possibly incomplete) description of a target task in natural language; next, a pre-trained LM translates this task description into a state abstraction function that masks out irrelevant features; finally, an imitation policy is trained using a small number of demonstrations and LGA-generated abstract states. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time, and that these abstractions improve generalization and robustness in the presence of spurious correlations and ambiguous specifications. We illustrate the utility of the learned abstractions on mobile manipulation tasks with a Spot robot.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑