使用主动学习最大化BCI人的反馈
Maximizing BCI Human Feedback using Active Learning
摘要：
\textit{Learning from Human Feedback}的最新进展提供了一种通过非专家人类的输入来训练机器人代理的有效方法，而无需专门设计的奖励函数。然而，这种方法需要人在机器人学习过程中在场并专心提供评价性反馈。此外，所需的反馈量随着任务难度的增加而增加，并且由于疲劳，人类反馈的质量可能会随着时间的推移而下降。为了克服这些限制并能够学习更多复杂性更高的机器人任务，需要最大限度地提高接收到的昂贵反馈的质量并减少所需的人类认知参与量。在这项工作中，我们提出了一种方法，使用主动学习智能地选择查询的人类主管的机器人的不确定性的基础上，有效地减少了学习给定的任务所需的反馈量。我们还使用了一种新的多缓冲系统，以提高反馈噪声的鲁棒性和防止灾难性遗忘的机器人学习的发展。与以前的方法相比，这使得使用更少的人类反馈来学习更复杂的任务成为可能。我们证明了我们提出的方法的实用性，机器人手臂达到的任务，机器人学会在3D中到达一个位置，而不会与障碍物碰撞。与以前不使用主动学习的方法相比，我们的方法能够更快地学习这项任务，减少人类反馈和认知参与。
英文摘要：
Recent advancements in \textit{Learning from Human Feedback} present an effective way to train robot agents via inputs from non-expert humans, without a need for a specially designed reward function. However, this approach needs a human to be present and attentive during robot learning to provide evaluative feedback. In addition, the amount of feedback needed grows with the level of task difficulty and the quality of human feedback might decrease over time because of fatigue. To overcome these limitations and enable learning more robot tasks with higher complexities, there is a need to maximize the quality of expensive feedback received and reduce the amount of human cognitive involvement required. In this work, we present an approach that uses active learning to smartly choose queries for the human supervisor based on the uncertainty of the robot and effectively reduces the amount of feedback needed to learn a given task. We also use a novel multiple buffer system to improve robustness to feedback noise and guard against catastrophic forgetting as the robot learning evolves. This makes it possible to learn tasks with more complexity using lesser amounts of human feedback compared to previous methods. We demonstrate the utility of our proposed method on a robot arm reaching task where the robot learns to reach a location in 3D without colliding with obstacles. Our approach is able to learn this task faster, with less human feedback and cognitive involvement, compared to previous methods that do not use active learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
