人机交互中语义驱动消歧的模型无关方法
A Model-Agnostic Approach for Semantically Driven Disambiguation in Human-Robot Interaction
AI总结:
本文提出一种模型无关的语义驱动消歧方法,利用知识嵌入模型和迭代预测的信息丰富澄清机制,解决人机交互中欠明确指令导致的物体搜索歧义问题,使机器人在首次尝试即可高效定位目标。
摘要:
在人机交互中,歧义是不可避免的,尤其是当机器人在大型共享空间中遵循用户指令时。例如,如果用户要求机器人在家庭环境中使用欠明确的指令寻找物体,该物体可能因缺失的因素而处于多个位置。例如,一个碗可能在厨房橱柜或餐桌上,这取决于它是干净还是脏的、满的还是空的,以及周围是否有其他物体。以往关于物体搜索的研究假设查询的物体对机器人立即可见,或使用单次推理预测物体位置,这对于歧义或部分理解的指令很可能会失败。本文聚焦这些差距,提出了一种新颖的模型无关方法,利用语义驱动的澄清来增强机器人在更少尝试中定位查询物体的能力。具体而言,我们利用不同的知识嵌入模型,并在出现歧义时,我们提出了一种信息丰富的澄清方法,该方法遵循迭代预测过程。对我们方法的用户实验评估表明,我们的方法适用于不同的自定义语义编码器以及LLMs,且信息丰富的澄清提高了性能,使机器人能够在首次尝试时定位物体。用户实验数据公开于 https://github.com/IrmakDogan/ExpressionDataset。
英文摘要:
Ambiguities are inevitable in human-robot interaction, especially when a robot follows user instructions in a large, shared space. For example, if a user asks the robot to find an object in a home environment with underspecified instructions, the object could be in multiple locations depending on missing factors. For instance, a bowl might be in the kitchen cabinet or on the dining room table, depending on whether it is clean or dirty, full or empty, and the presence of other objects around it. Previous works on object search have assumed that the queried object is immediately visible to the robot or have predicted object locations using one-shot inferences, which are likely to fail for ambiguous or partially understood instructions. This paper focuses on these gaps and presents a novel model-agnostic approach leveraging semantically driven clarifications to enhance the robot's ability to locate queried objects in fewer attempts. Specifically, we leverage different knowledge embedding models, and when ambiguities arise, we propose an informative clarification method, which follows an iterative prediction process. The user experiment evaluation of our method shows that our approach is applicable to different custom semantic encoders as well as LLMs, and informative clarifications improve performances, enabling the robot to locate objects on its first attempts. The user experiment data is publicly available at https://github.com/IrmakDogan/ExpressionDataset.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。