arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.00696cs.CL
收藏

基于关系网络的空间参考的鲁棒可解释性Grounding

Robust and Interpretable Grounding of Spatial References with Relation Networks

Princeton University(普林斯顿大学);University of Massachusetts, Amherst(马萨诸塞大学阿默斯特分校)

Tsung-Yen Yang, Andrew S. Lan, Karthik Narasimhan

摘要:

学习自然语言中的空间参考表示是自主导航和机器人操作等任务的关键挑战。最近的工作已经调查了各种神经架构学习多模态表示的空间概念。然而,缺乏显式推理的实体,使这些方法容易受到噪音的输入文本或状态观察。在本文中,我们开发了有效的模型来理解文本中的空间参考,这些模型是鲁棒的和可解释的,而不会牺牲性能。我们设计了一个文本条件的\textit{relation network},其参数动态计算的跨模态注意力模块捕捉实体之间的细粒度的空间关系。这种设计选择提供了学习的中间输出的可解释性。三个任务的实验表明,我们的模型实现了卓越的性能,与最先进的系统相比,预测目标位置提高了17%,鲁棒性提高了15%。

英文摘要:

Learning representations of spatial references in natural language is a key challenge in tasks like autonomous navigation and robotic manipulation. Recent work has investigated various neural architectures for learning multi-modal representations for spatial concepts. However, the lack of explicit reasoning over entities makes such approaches vulnerable to noise in input text or state observations. In this paper, we develop effective models for understanding spatial references in text that are robust and interpretable, without sacrificing performance. We design a text-conditioned \textit{relation network} whose parameters are dynamically computed with a cross-modal attention module to capture fine-grained spatial relations between entities. This design choice provides interpretability of learned intermediate outputs. Experiments across three tasks demonstrate that our model achieves superior performance, with a 17\% improvement in predicting goal locations and a 15\% improvement in robustness compared to state-of-the-art systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑