arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1706.00355cs.AI
收藏

多模式指令中的接地符号

Grounding Symbols in Multi-Modal Instructions

School of Informatics(信息学院);The University of Edinburgh(爱丁堡大学)

Yordan Hristov, Svetlin Penkov, Alex Lascarides, Subramanian Ramamoorthy

摘要:

随着机器人开始在半结构化环境中与人类共存,需要理解包含丰富可变性的指令-例如,学习在物理世界中接地符号。实际上,这个任务必须处理由特定用户对术语的上下文意义分配组成的小数据集。我们提出了一种处理跨模态输入的原始流的方法,即,语言指令、场景的视觉感知以及3D眼动追踪注视的并发追踪--以产生与高级概念相对应关联的对象分割。为了测试我们的框架,我们提出了一个桌面对象操作的情况下的实验。我们的研究结果表明,我们的模型从少量的物理演示中学习用户对颜色和形状的概念,并概括为识别单词的新颖组合的物理对象。

英文摘要:

As robots begin to cohabit with humans in semi-structured environments, the need arises to understand instructions involving rich variability---for instance, learning to ground symbols in the physical world. Realistically, this task must cope with small datasets consisting of a particular users' contextual assignment of meaning to terms. We present a method for processing a raw stream of cross-modal input---i.e., linguistic instructions, visual perception of a scene and a concurrent trace of 3D eye tracking fixations---to produce the segmentation of objects with a correspondent association to high-level concepts. To test our framework we present experiments in a table-top object manipulation scenario. Our results show our model learns the user's notion of colour and shape from a small number of physical demonstrations, generalising to identifying physical referents for novel combinations of the words.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑