使用视觉问答模型从基于视觉基础的语言中学习功能词的含义
Learning the meanings of function words from grounded language using a visual question answering model
AI总结：
本文研究视觉问答模型如何从基于视觉基础的语言中学习功能词含义，发现循环模型能通过非符号化统计学习掌握逻辑、空间和数值功能词语义，且学习难度受输入频率影响。
摘要：
解释像“or”、“behind”或“more”这样看似简单的功能词可能需要逻辑、数值和关系推理。儿童是如何学习这些词的？先前的习得理论通常依赖于假设一种先天知识的基础。然而，最近基于神经网络的视觉问答模型显然能够学会将功能词作为回答关于复杂视觉场景问题的一部分。在本文中，我们研究了这些模型学习了关于功能词的哪些内容，希望能更好地理解模型和儿童如何学习这些词的含义。我们表明，在基于视觉基础的语言上训练的循环模型能够为需要空间和数值推理的功能词学习梯度语义。此外，我们发现这些模型能够在没有任何逻辑推理先验知识的情况下学习逻辑连接词and和or的含义，以及它们在解释语言时对替代表达敏感的早期证据。最后，我们表明，单词学习难度取决于模型输入中的频率。我们的发现提供了概念验证证据，即通过使用非符号化的一般统计学习算法，在没有任何语言意义先验知识的情况下，可以在基于视觉基础的上下文中学习功能词的细微解释。
英文摘要：
Interpreting a seemingly-simple function word like "or", "behind", or "more" can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural-network based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learnt by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spatial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives and and or without any prior knowledge of logical reasoning, as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on frequency in models' input. Our findings offer proof-of-concept evidence that it is possible to learn the nuanced interpretations of function words in visually grounded context by using non-symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
