像双语婴儿一样：视觉基础双语语言模型的优势
Like a bilingual baby: The advantage of visually grounding a bilingual language model
AI总结：
研究旨在探讨视觉基础对双语语言模型的影响，通过在英西双语图像及字幕上训练LSTM模型，发现视觉基础能提升模型对语义相似性的理解及降低困惑度，为视觉基础语言模型优势提供证据，指出需更多自然语言数据。
摘要：
与大多数神经语言模型不同，人类在丰富、多感官且通常是多语言的环境中学习语言。当前语言模型通常无法完全捕捉多语言语言使用的复杂性。我们在来自MS-COCO-ES的英语和西班牙语图像及字幕上训练长短期记忆（LSTM）语言模型。发现视觉基础提高了模型对语言内和跨语言语义相似性的理解并降低了困惑度，但对抽象词无显著优势。结果为视觉基础语言模型的优势提供了更多证据，表明需要来自多语言使用者的更自然语言数据和具有感知基础的多语言数据集。
英文摘要：
Unlike most neural language models, humans learn language in a rich, multi-sensory and, often, multi-lingual environment. Current language models typically fail to fully capture the complexities of multilingual language use. We train an LSTM language model on images and captions in English and Spanish from MS-COCO-ES. We find that the visual grounding improves the model's understanding of semantic similarity both within and across languages and improves perplexity. However, we find no significant advantage of visual grounding for abstract words. Our results provide additional evidence of the advantages of visually grounded language models and point to the need for more naturalistic language data from multilingual speakers and multilingual datasets with perceptual grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
