从更丰富的人类指导中学习：用特征搜索增强基于比较的学习
Learning from Richer Human Guidance: Augmenting Comparison-Based Learning with Feature Queries
摘要：
我们专注于学习机器人所需的目标函数。虽然轨迹演示可以提供所需目标的大量信息，但用户也很难提供。回答比较查询，询问两个轨迹中的哪一个更好，对用户来说更容易，并且已经成为一种有效的替代方案。不幸的是，比较的信息量要少得多。我们认为，有更丰富的信息，用户可以很容易地提供，机器人应该利用。我们专注于增强与特征查询的比较，并引入了一个统一的形式主义，用于将所有答案视为对真正期望奖励的观察。我们推导出一个主动查询选择算法，并测试这些查询在模拟和真实用户。我们发现，更丰富的、功能增强的查询可以更快地提取更多信息，从而使机器人在行为上更好地匹配用户偏好。
英文摘要：
We focus on learning the desired objective function for a robot. Although trajectory demonstrations can be very informative of the desired objective, they can also be difficult for users to provide. Answers to comparison queries, asking which of two trajectories is preferable, are much easier for users, and have emerged as an effective alternative. Unfortunately, comparisons are far less informative. We propose that there is much richer information that users can easily provide and that robots ought to leverage. We focus on augmenting comparisons with feature queries, and introduce a unified formalism for treating all answers as observations about the true desired reward. We derive an active query selection algorithm, and test these queries in simulation and on real users. We find that richer, feature-augmented queries can extract more information faster, leading to robots that better match user preferences in their behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
