arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.01995cs.HCcs.RO
收藏

以下是我学到的:提出揭示奖励学习的问题

Here's What I've Learned: Asking Questions that Reveal Reward Learning

Virginia Tech(弗吉尼亚理工大学)

Soheil Habibian, Ananth Jonnavittula, Dylan P. Losey

摘要:

机器人可以通过提问向人类学习。在这些问题中,机器人展示了一些不同的行为,并询问人类最喜欢的行为。但是机器人应该如何选择要问哪些问题呢?今天的机器人优化了信息问题,尽可能有效地积极探索人类的偏好。但是,虽然从机器人的角度来看,信息性问题是有意义的,但人类旁观者往往会发现它们是武断和误导的。在本文中,我们从人类的角度正式的主动偏好为基础的学习。我们假设,从人类的角度来看,机器人的问题揭示了机器人已经学到了什么和没有学到什么。我们的洞察力使机器人能够使用问题,使他们的学习过程对人类操作员透明。我们开发并测试了一个模型,机器人可以利用这个模型将他们提出的问题与这些问题所揭示的信息联系起来。然后,我们引入了一个权衡信息和揭示的问题,考虑人类和机器人的角度:一个机器人,优化这种权衡积极收集来自人类的信息,同时保持人类最新的与它所学到的。我们通过模拟、在线调查和亲自用户研究来评估我们的方法。我们的用户研究和结果的视频可在这里:https://youtu.be/tC6y_jHN7Vw.

英文摘要:

Robots can learn from humans by asking questions. In these questions the robot demonstrates a few different behaviors and asks the human for their favorite. But how should robots choose which questions to ask? Today's robots optimize for informative questions that actively probe the human's preferences as efficiently as possible. But while informative questions make sense from the robot's perspective, human onlookers often find them arbitrary and misleading. In this paper we formalize active preference-based learning from the human's perspective. We hypothesize that -- from the human's point-of-view -- the robot's questions reveal what the robot has and has not learned. Our insight enables robots to use questions to make their learning process transparent to the human operator. We develop and test a model that robots can leverage to relate the questions they ask to the information these questions reveal. We then introduce a trade-off between informative and revealing questions that considers both human and robot perspectives: a robot that optimizes for this trade-off actively gathers information from the human while simultaneously keeping the human up to date with what it has learned. We evaluate our approach across simulations, online surveys, and in-person user studies. Videos of our user studies and results are available here: https://youtu.be/tC6y_jHN7Vw.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑