基于语义地图监督的导航视觉表示学习
Learning Navigational Visual Representations with Semantic Map Supervision
AI总结：
针对现有导航视觉表示忽略空间关系的问题，提出Ego²-Map导航专属表示学习方法，通过自视角与语义地图对比迁移地图信息，在目标物体导航、视觉-语言导航任务上取得性能提升与SOTA结果。
摘要：
感知环境的语义与空间结构是家用机器人实现视觉导航的核心能力。但现有研究大多仅采用针对独立图像分类预训练、或经自监督学习适配室内导航场景的视觉骨干网络，忽略了对导航学习至关重要的空间关联关系。受人类导航时会在脑中构建兼具语义与空间意义的认知地图的行为启发，本文提出一种全新的导航专属视觉表示学习方法——Ego$^2$-Map（自视角-语义地图对比学习），通过智能体自视角视图与语义地图的对比完成表示学习。该方法以视觉Transformer作为骨干编码器，采用从大规模Habitat-Matterport3D环境中采集的数据训练模型。Ego$^2$-Map学习可将语义地图中包含物体、结构、转移关系在内的紧凑丰富信息，迁移至智能体的自视角表示以服务导航任务。实验表明，搭载本文所提学习表示的智能体在目标物体导航任务上的表现优于近年的视觉预训练方法。此外，该表示在连续环境的视觉-语言导航任务中，针对高层与底层动作空间均实现显著性能提升，在测试服务器上取得47% SR（成功率）与41% SPL（路径长度加权成功率）的全新最优结果。
英文摘要：
Being able to perceive the semantics and the spatial structure of the environment is essential for visual navigation of a household robot. However, most existing works only employ visual backbones pre-trained either with independent images for classification or with self-supervised learning methods to adapt to the indoor navigation domain, neglecting the spatial relationships that are essential to the learning of navigation. Inspired by the behavior that humans naturally build semantically and spatially meaningful cognitive maps in their brains during navigation, in this paper, we propose a novel navigational-specific visual representation learning method by contrasting the agent's egocentric views and semantic maps (Ego$^2$-Map). We apply the visual transformer as the backbone encoder and train the model with data collected from the large-scale Habitat-Matterport3D environments. Ego$^2$-Map learning transfers the compact and rich information from a map, such as objects, structure and transition, to the agent's egocentric representations for navigation. Experiments show that agents using our learned representations on object-goal navigation outperform recent visual pre-training methods. Moreover, our representations significantly improve vision-and-language navigation in continuous environments for both high-level and low-level action spaces, achieving new state-of-the-art results of 47% SR and 41% SPL on the test server.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
