arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.05052cs.RO
收藏

基于VQA并通过遗传算法优化的机器人状态识别

VQA-based Robotic State Recognition Optimized with Genetic Algorithm

The University of Tokyo(东京大学)

Kento Kawaharazuka, Yoshiki Obinata, Naoaki Kanazawa, Kei Okada, Masayuki Inaba

AI总结:

针对机器人状态识别问题,本文提出在预训练视觉-语言模型中使用VQA,并通过遗传算法优化问题组合,实现对冰箱门、显示器、透明门及水状态的有效识别。

摘要:

机器人对物体和环境的状态识别已经通过多种方式开展。在大多数情况下,这是通过处理点云、利用带标注的图像进行学习以及使用专用传感器来执行的。相比之下,在本研究中,我们提出一种状态识别方法,该方法将视觉问答(VQA)应用于在大规模数据集上训练的预训练视觉-语言模型(PTVLM)。通过使用VQA,可以用口语直观地描述机器人状态识别。另一方面,对同一事件进行提问存在多种可能方式,而状态识别的性能会因问题不同而有所差异。因此,为了提高使用VQA进行状态识别的性能,我们采用遗传算法搜索合适的问题组合。我们表明,我们的系统不仅能够识别冰箱门的开/关和显示器的开/关,还能够识别一直难以识别的透明门的开/关以及水的状态。

英文摘要:

State recognition of objects and environment in robots has been conducted in various ways. In most cases, this is executed by processing point clouds, learning images with annotations, and using specialized sensors. In contrast, in this study, we propose a state recognition method that applies Visual Question Answering (VQA) in a Pre-Trained Vision-Language Model (PTVLM) trained from a large-scale dataset. By using VQA, it is possible to intuitively describe robotic state recognition in the spoken language. On the other hand, there are various possible ways to ask about the same event, and the performance of state recognition differs depending on the question. Therefore, in order to improve the performance of state recognition using VQA, we search for an appropriate combination of questions using a genetic algorithm. We show that our system can recognize not only the open/closed of a refrigerator door and the on/off of a display, but also the open/closed of a transparent door and the state of water, which have been difficult to recognize.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑