Mind the Error! 视觉语言导航中指令错误的检测与定位
Mind the Error! Detection and Localization of Instruction Errors in Vision-and-Language Navigation
AI总结：
针对现有VLN-CE方法假设指令准确、面对人类错误指令鲁棒性差的问题，本文提出含多类指令错误的基准数据集，定义错误检测定位任务并给出评估协议，同时提出基于跨模态transformer的最优方法，还在现有常用数据集中发现了错误。
摘要：
连续环境下的视觉语言导航（Vision-and-Language Navigation in Continuous Environments, VLN-CE）是最直观但也最具挑战性的具身人工智能任务之一。智能体需要遵循一系列自然语言指令，通过执行一组底层动作导航至目标位置。现有文献中所有的VLN-CE方法都假设语言指令是准确无误的。但在实际场景中，人类给出的指令在描述空间环境时，可能会因记忆不准确或认知混淆而包含错误。当前的VLN-CE基准并未覆盖这一场景，导致现有最先进的VLN-CE方法在面对人类用户的错误指令时鲁棒性很差。\n我们首次提出了一个全新的基准数据集，该数据集结合人类可能出错的原因，引入了多种类型的指令错误。这一基准为评估连续环境下VLN系统的鲁棒性提供了重要参考。我们在该基准上测试现有最先进的VLN-CE方法时，观察到成功率出现了显著下降（降幅最高达25%）。\n此外，我们正式定义了“指令错误检测与定位”任务，并在我们的基准数据集之上建立了评估协议。我们还提出了一种基于跨模态transformer架构的有效方法，与基线方法相比，该方法在错误检测与定位任务上取得了最优性能。令人意外的是，我们提出的方法还在两个常用VLN-CE数据集——即R2R-CE和RxR-CE——的验证集中发现了错误，证明了我们的技术在其他任务中的实用价值。代码与数据集可访问https://intelligolabs.github.io/R2RIE-CE获取。
英文摘要：
Vision-and-Language Navigation in Continuous Environments (VLN-CE) is one of the most intuitive yet challenging embodied AI tasks. Agents are tasked to navigate towards a target goal by executing a set of low-level actions, following a series of natural language instructions. All VLN-CE methods in the literature assume that language instructions are exact. However, in practice, instructions given by humans can contain errors when describing a spatial environment due to inaccurate memory or confusion. Current VLN-CE benchmarks do not address this scenario, making the state-of-the-art methods in VLN-CE fragile in the presence of erroneous instructions from human users. For the first time, we propose a novel benchmark dataset that introduces various types of instruction errors considering potential human causes. This benchmark provides valuable insight into the robustness of VLN systems in continuous environments. We observe a noticeable performance drop (up to -25%) in Success Rate when evaluating the state-of-the-art VLN-CE methods on our benchmark. Moreover, we formally define the task of Instruction Error Detection and Localization, and establish an evaluation protocol on top of our benchmark dataset. We also propose an effective method, based on a cross-modal transformer architecture, that achieves the best performance in error detection and localization, compared to baselines. Surprisingly, our proposed method has revealed errors in the validation set of the two commonly used datasets for VLN-CE, i.e., R2R-CE and RxR-CE, demonstrating the utility of our technique in other tasks. Code and dataset available at https://intelligolabs.github.io/R2RIE-CE
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
