从探索性动作中进行对比学习：利用自然交互实现偏好引导
Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation
AI总结：
本文提出 CLEA 方法，利用用户在机器人定制中的探索性动作作为自然标注信号，通过对比学习获取与用户偏好对齐的轨迹特征，并在用户研究中验证其优于自监督特征。
摘要：
人们对机器人如何表现有着多种多样的偏好。为了理解并推理这些偏好，机器人旨在学习一个奖励函数，用以描述机器人行为与用户偏好的对齐程度。良好的机器人行为表征可以显著减少用户教导机器人其偏好所需的时间和精力。然而，指定这些表征——即机器人行为的哪些“特征”对用户重要——仍然是一个困难的问题；从原始数据中学习到的特征缺乏语义含义，而从用户数据中学习到的特征则需要用户参与繁琐的标注过程。我们的关键见解是：被要求定制机器人的用户具有通过探索性搜索产生标注的内在动机；他们会探索自己感兴趣的行为，并忽略不相关的行为。为了利用这一新颖的探索性动作数据来源，我们提出了从探索性动作中进行对比学习（CLEA），以学习与用户所关注特征相对齐的轨迹特征。我们从一个开放式信号设计活动（N=25）中用户使用 Kuri 机器人执行的探索性动作中学习 CLEA 特征，并通过第二项用户研究（N=42）使用另一组用户对 CLEA 特征进行评估。在引导用户偏好时，CLEA 特征在四个指标上优于自监督特征：完整性、简洁性、最小性和可解释性。
英文摘要：
People have a variety of preferences for how robots behave. To understand and reason about these preferences, robots aim to learn a reward function that describes how aligned robot behaviors are with a user's preferences. Good representations of a robot's behavior can significantly reduce the time and effort required for a user to teach the robot their preferences. Specifying these representations -- what "features" of the robot's behavior matter to users -- remains a difficult problem; Features learned from raw data lack semantic meaning and features learned from user data require users to engage in tedious labeling processes. Our key insight is that users tasked with customizing a robot are intrinsically motivated to produce labels through exploratory search; they explore behaviors that they find interesting and ignore behaviors that are irrelevant. To harness this novel data source of exploratory actions, we propose contrastive learning from exploratory actions (CLEA) to learn trajectory features that are aligned with features that users care about. We learned CLEA features from exploratory actions users performed in an open-ended signal design activity (N=25) with a Kuri robot, and evaluated CLEA features through a second user study with a different set of users (N=42). CLEA features outperformed self-supervised features when eliciting user preferences over four metrics: completeness, simplicity, minimality, and explainability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
