arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.13822cs.RO
收藏

通过基于偏好的动作表示学习实现人机交互中的个性化

Personalization in Human-Robot Interaction through Preference-based Action Representation Learning

Purdue University(普渡大学);College of Mechanical and Electrical Engineering, Beijing University of Chemical Technology (BUCT)(北京化工大学机电工程学院)

Ruiqi Wang, Dezhong Zhao, Dayoon Suh, Ziqin Yuan, Guohua Chen, Byung-Cheol Min

AI总结:

本文提出PbARL方法,通过基于预训练策略的动作表示学习实现高效人机交互个性化,在保持任务性能的同时提升人类反馈利用效率,并在基准测试和用户研究中验证了其优势。

摘要:

基于偏好的强化学习(PbRL)通过将人类偏好显式地整合到机器人学习过程中,已在人机交互(HRI)的个性化方面展现出显著前景。然而,现有做法通常需要从头训练个性化机器人策略,导致对人类反馈的利用效率低下。在本文中,我们提出了基于偏好的动作表示学习(PbARL),这是一种高效的微调方法,通过利用预训练机器人策略将通用任务结构与偏好解耦。PbARL并非直接用人类偏好微调预训练策略,而是将其作为动作表示学习任务的参考,该任务最大化预训练源域与目标用户偏好对齐域之间的互信息。该方法使机器人能够在保持原有任务性能的同时实现行为个性化,并消除了对源域大量先验信息的需求,从而提升了真实世界HRI场景中的效率和实用性。在Assistive Gym基准测试和一项真实世界用户研究(N=8)上的实证结果表明,与最先进的方法相比,我们的方法具有明显优势。

英文摘要:

Preference-based reinforcement learning (PbRL) has shown significant promise for personalization in human-robot interaction (HRI) by explicitly integrating human preferences into the robot learning process. However, existing practices often require training a personalized robot policy from scratch, resulting in inefficient use of human feedback. In this paper, we propose preference-based action representation learning (PbARL), an efficient fine-tuning method that decouples common task structure from preference by leveraging pre-trained robot policies. Instead of directly fine-tuning the pre-trained policy with human preference, PbARL uses it as a reference for an action representation learning task that maximizes the mutual information between the pre-trained source domain and the target user preference-aligned domain. This approach allows the robot to personalize its behaviors while preserving original task performance and eliminates the need for extensive prior information from the source domain, thereby enhancing efficiency and practicality in real-world HRI scenarios. Empirical results on the Assistive Gym benchmark and a real-world user study (N=8) demonstrate the benefits of our method compared to state-of-the-art approaches.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑