arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2404.09647cs.RO
收藏

辅助机器人中的物体实例检索:利用基于3D语义地图的多视角图像微调的SimSiam

Object Instance Retrieval in Assistive Robotics: Leveraging Fine-Tuned SimSiam with Multi-View Images Based on 3D Semantic Map

Ritsumeikan University(立命馆大学);Soka University(早稻田大学);Kyoto University(京都大学)

Taichi Sakaguchi, Akira Taniguchi, Yoshinobu Hagiwara, Lotfi El Hafi, Shoichi Hasegawa, Tadahiro Taniguchi

AI总结:

针对辅助机器人InstanceImageNav任务中多视角下物体实例难识别的问题,提出SimView方法,结合3D语义地图多视角图像与SimSiam自监督学习现场训练模型,在Habitat Matterport 3D上验证较CLIP准确率提升1.7倍。

摘要:

在日常生活中为人类提供协助的机器人应具备在环境中定位与用户需求物体匹配的特定物体实例的能力。该任务被称为实例特定图像目标导航(InstanceImageNav),要求模型能够区分同一类别下的不同物体实例。机器人领域的一个重大挑战是,当机器人从不同3D视角观察同一物体时,其外观可能存在显著差异,导致难以准确识别和定位。本文提出了一种名为SimView的方法,该方法利用基于环境3D语义地图的多视角图像,并借助SimSiam(孪生简单网络)进行自监督学习,以在现场训练实例识别模型。我们通过对真实家居环境扫描构建的照片级真实感模拟器Habitat Matterport 3D验证了该方法的有效性。结果表明,与用于物体搜索的预训练多模态对比学习方法对比语言-图像预训练(CLIP)相比,任务准确率提升了1.7倍。这一提升凸显了我们提出的微调方法在增强辅助机器人InstanceImageNav任务性能方面的优势。项目网站为https://emergentsystemlabstudent.github.io/MultiViewRetrieve/。

英文摘要:

Robots that assist humans in their daily lives should be able to locate specific instances of objects in an environment that match a user's desired objects. This task is known as instance-specific image goal navigation (InstanceImageNav), which requires a model that can distinguish different instances of an object within the same class. A significant challenge in robotics is that when a robot observes the same object from various 3D viewpoints, its appearance may differ significantly, making it difficult to recognize and locate accurately. In this paper, we introduce a method called SimView, which leverages multi-view images based on a 3D semantic map of an environment and self-supervised learning using SimSiam to train an instance-identification model on-site. The effectiveness of our approach was validated using a photorealistic simulator, Habitat Matterport 3D, created by scanning actual home environments. Our results demonstrate a 1.7-fold improvement in task accuracy compared with contrastive language-image pre-training (CLIP), a pre-trained multimodal contrastive learning method for object searching. This improvement highlights the benefits of our proposed fine-tuning method in enhancing the performance of assistive robots in InstanceImageNav tasks. The project website is https://emergentsystemlabstudent.github.io/MultiViewRetrieve/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑