arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.03081cs.ROcs.AIcs.LG
收藏

偏好条件下的语言引导抽象

Preference-Conditioned Language-Guided Abstraction

MIT(麻省理工学院);Boston Dynamics AI Institute(波士顿动力人工智能研究所);Princeton(普林斯顿大学)

Andi Peng, Andreea Bobu, Belinda Z. Li, Theodore R. Sumers, Ilia Sucholutsky, Nishanth Kumar, Thomas L. Griffiths, Julie A. Shah

AI总结:

本文提出利用语言模型在行为变化时查询用户潜在偏好,构建偏好条件的状态抽象,以提高机器人从示范学习的泛化性,并在模拟、用户研究和真实机器人上验证有效性。

摘要:

从示范中学习是用户教机器人的常见方式,但容易产生虚假的特征相关性。最近的工作从语言中构建状态抽象,即包含任务相关特征的视觉表示,作为一种进行更可泛化学习的方式。然而,这些抽象也依赖于用户对任务中重要内容的偏好,这可能难以描述或无法仅用语言详尽指定。我们如何构建抽象以捕获这些潜在偏好?我们观察到人类行为方式揭示了他们如何看待世界。我们的关键见解是,人类行为的变化告诉我们,人类看待世界的方式存在偏好差异,即他们的状态抽象。在这项工作中,我们提出使用语言模型(LM)在已知行为变化发生时直接查询这些偏好。在我们的框架中,我们以两种方式使用LM:首先,给定任务文本描述和状态间行为变化的知识,我们查询LM以获取可能的隐藏偏好;其次,给定最可能的偏好,我们查询LM以构建状态抽象。在此框架中,LM在不确定自身估计时也能直接询问人类。我们通过模拟实验、用户研究以及在真实Spot机器人上执行移动操作任务,展示了我们的框架构建有效偏好条件抽象的能力。

英文摘要:

Learning from demonstrations is a common way for users to teach robots, but it is prone to spurious feature correlations. Recent work constructs state abstractions, i.e. visual representations containing task-relevant features, from language as a way to perform more generalizable learning. However, these abstractions also depend on a user's preference for what matters in a task, which may be hard to describe or infeasible to exhaustively specify using language alone. How do we construct abstractions to capture these latent preferences? We observe that how humans behave reveals how they see the world. Our key insight is that changes in human behavior inform us that there are differences in preferences for how humans see the world, i.e. their state abstractions. In this work, we propose using language models (LMs) to query for those preferences directly given knowledge that a change in behavior has occurred. In our framework, we use the LM in two ways: first, given a text description of the task and knowledge of behavioral change between states, we query the LM for possible hidden preferences; second, given the most likely preference, we query the LM to construct the state abstraction. In this framework, the LM is also able to ask the human directly when uncertain about its own estimate. We demonstrate our framework's ability to construct effective preference-conditioned abstractions in simulated experiments, a user study, as well as on a real Spot robot performing mobile manipulation tasks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑