哪条路才是“正确”的？：揭示视觉-语言导航模型的局限性
Which way is `right'?: Uncovering limitations of Vision-and-Language Navigation model
AI总结：
本文通过掩码实验揭示视觉-语言导航模型过度依赖名词而忽视空间方向线索的局限性，并提出两种训练方法加以缓解。
摘要：
视觉-语言导航（VLN）这一具有挑战性的任务要求具身智能体遵循自然语言指令到达目标位置或物体（例如“沿着走廊走，在钢琴处左转”）。智能体要成功完成该任务，必须能够将指令中提到的物体（例如“钢琴”）与视觉场景对应起来，并将方向性短语（例如“左转”）落实为具体动作。本文提出以下问题——空间和方向性语言线索在多大程度上影响着导航模型的决策？我们设计了一系列简单的掩码实验，以考察模型对指令不同部分的依赖程度。令人惊讶的是，我们发现某些表现最优的模型仅依赖指令中的名词词元。针对这一令人担忧的局限性，我们提出了两种训练方法来加以缓解。
英文摘要：
The challenging task of Vision-and-Language Navigation (VLN) requires embodied agents to follow natural language instructions to reach a goal location or object (e.g. `walk down the hallway and turn left at the piano'). For agents to complete this task successfully, they must be able to ground objects referenced into the instruction (e.g.`piano') into the visual scene as well as ground directional phrases (e.g.`turn left') into actions. In this work we ask the following question -- to what degree are spatial and directional language cues informing the navigation model's decisions? We propose a series of simple masking experiments to inspect the model's reliance on different parts of the instruction. Surprisingly we uncover that certain top performing models rely only on the noun tokens of the instructions. We propose two training methods to alleviate this concerning limitation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
