用于从观察中学习的言语注意焦点系统
Verbal Focus-of-Attention System for Learning-from-Observation
摘要：
从观察中学习（LfO）框架旨在将人类演示映射到机器人，以减少编程工作。为此，LfO系统将人类演示编码为机器人的一系列执行单元，这些单元被称为任务模型。虽然以前的研究已经提出了成功的任务模型编码器，很少有讨论如何引导一个任务模型编码器在场景中的时空噪声，如杂乱的物体或无关的人体运动。受言语指令引导观察者视觉注意力的功能的启发，我们提出了一种言语注意力集中（FoA）系统（即，时空滤波器）来引导任务模型编码器。对于对象操作，系统首先从口头指令识别目标对象的名称及其属性。该信息用作何处查看FoA过滤器，以限制演示中存在目标对象的区域。然后，系统检测在过滤区域中发生的抓握和释放的时间。计时用作何时查看FoA过滤器，以限制对象操作的时间。最后，任务模型编码器通过采用FoA过滤器来识别任务模型。我们通过与现有的动作定位网络进行比较，证明了言语FoA在衰减时空噪声方面的鲁棒性。本研究的贡献如下：（1）提出了一个语言的FoA为LfO，（2）设计一个算法来计算FoA过滤器从语言输入，和（3）通过比较它与最先进的视觉系统的语言FoA在定位一个动作的有效性。
英文摘要：
The learning-from-observation (LfO) framework aims to map human demonstrations to a robot to reduce programming effort. To this end, an LfO system encodes a human demonstration into a series of execution units for a robot, which are referred to as task models. Although previous research has proposed successful task-model encoders, there has been little discussion on how to guide a task-model encoder in a scene with spatio-temporal noises, such as cluttered objects or unrelated human body movements. Inspired by the function of verbal instructions guiding an observer's visual attention, we propose a verbal focus-of-attention (FoA) system (i.e., spatio-temporal filters) to guide a task-model encoder. For object manipulation, the system first recognizes the name of a target object and its attributes from verbal instructions. The information serves as a where-to-look FoA filter to confine the areas in which the target object existed in the demonstration. The system then detects the timings of grasp and release that occurred in the filtered areas. The timings serve as a when-to-look FoA filter to confine the period of object manipulation. Finally, a task-model encoder recognizes the task models by employing FoA filters. We demonstrate the robustness of the verbal FoA in attenuating spatio-temporal noises by comparing it with an existing action localization network. The contributions of this study are as follows: (1) to propose a verbal FoA for LfO, (2) to design an algorithm to calculate FoA filters from verbal input, and (3) to demonstrate the effectiveness of a verbal FoA in localizing an action by comparing it with a state-of-the-art vision system.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
