视觉能加速神经语言学习者的层级泛化吗？
Does Vision Accelerate Hierarchical Generalization in Neural Language Learners?
AI总结：
本研究通过刺激贫乏范式实验，探究视觉信息对语言模型句法泛化的影响，发现清晰的语言-视觉对齐能加速泛化，否则无效，并强调需额外信号增强跨模态对齐。
摘要：
从语言习得的角度来看，神经语言模型（LMs）的数据效率可以说低于人类。一个基本的问题是为什么会出现这种人类与语言模型之间的差距。本研究探讨了基于实体的语言习得的优势，具体而言，是视觉信息——人类在语言习得过程中通常可以依赖而语言模型大多无法获取的信息——对语言模型句法泛化的影响。我们的实验遵循刺激贫乏范式，在两种场景下（使用人工图像与自然图像）进行，结果表明，如果输入中语言与视觉成分之间的对齐清晰，视觉数据的获取确实有助于语言模型的句法泛化；但如果对齐不清晰，视觉输入则无济于事。这凸显了需要额外的偏见或信号（如相互注视）来增强跨模态对齐，从而使多模态语言模型能够实现高效的句法泛化。
英文摘要：
Neural language models (LMs) are arguably less data-efficient than humans from a language acquisition perspective. One fundamental question is why this human-LM gap arises. This study explores the advantage of grounded language acquisition, specifically the impact of visual information -- which humans can usually rely on but LMs largely do not have access to during language acquisition -- on syntactic generalization in LMs. Our experiments, following the poverty of stimulus paradigm under two scenarios (using artificial vs. naturalistic images), demonstrate that if the alignments between the linguistic and visual components are clear in the input, access to vision data does help with the syntactic generalization of LMs, but if not, visual input does not help. This highlights the need for additional biases or signals, such as mutual gaze, to enhance cross-modal alignment and enable efficient syntactic generalization in multimodal LMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
