World-to-Words：通过视觉-语言模型中的快速映射实现基于真实世界的开放词汇习得
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models
AI总结：
本文提出GOVA框架和OctoBERT模型，通过在图像-文本对上以接地为目标的预训练，验证视觉-语言模型能更快速稳健地学习新词汇，揭示了接地能力对新词学习的自举作用。
摘要：
将语言单元与其在物理世界中的指称对象相连接的能力，即所谓“接地”（grounding），对于学习和理解词汇的接地含义至关重要。虽然人类在新词学习中表现出快速映射能力，但现代视觉-语言模型是否能够真正用接地含义来表示语言，以及接地如何进一步促进新词学习，仍不清楚。为此，我们提出了接地开放词汇习得（Grounded Open Vocabulary Acquisition，GOVA）框架，以考察开放世界语言学习中的接地与自举（bootstrapping）问题。作为初步尝试，我们提出了面向对象的BERT（OctoBERT），这是一种新颖的视觉接地语言模型，通过在图像-文本对上进行预训练，将接地作为显式目标。通过大量实验和分析，我们证明OctoBERT是一个更连贯、更快速的接地词汇学习器，并且预训练阶段获得的接地能力有助于模型更快速、更稳健地学习未见过的词汇。我们的代码可在 https://github.com/sled-group/world-to-words 获取。
英文摘要：
The ability to connect language units to their referents in the physical world, referred to as grounding, is crucial to learning and understanding grounded meanings of words. While humans demonstrate fast mapping in new word learning, it remains unclear whether modern vision-language models can truly represent language with their grounded meanings and how grounding may further bootstrap new word learning. To this end, we introduce Grounded Open Vocabulary Acquisition (GOVA) to examine grounding and bootstrapping in open-world language learning. As an initial attempt, we propose object-oriented BERT (OctoBERT), a novel visually-grounded language model by pre-training on image-text pairs highlighting grounding as an objective. Through extensive experiments and analysis, we demonstrate that OctoBERT is a more coherent and fast grounded word learner, and that the grounding ability acquired during pre-training helps the model to learn unseen words more rapidly and robustly. Our code is available at https://github.com/sled-group/world-to-words
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
