视觉-语言训练有助于部署分类学知识，但并未从根本上改变它
Vision-and-Language Training Helps Deploy Taxonomic Knowledge but Does Not Fundamentally Alter It
AI总结：
本研究发现视觉-语言训练虽未从根本上改变语言模型的分类学知识本身，但能改善模型在特定纯文本任务中部署该知识的能力。
摘要：
视觉-语言（VL）训练是否以有意义的方式改变了语言模型的语言表示？文献中的大多数结果表明，无论在行为还是表示层面，差异都不一致或微乎其微。在本研究中，我们从VL训练可能产生显著影响的领域是词汇-概念知识，特别是其分类学组织这一假设出发。通过比较纯文本语言模型与其经过VL训练的对应模型的最小对，我们首先表明，在一个需要理解问题中提及概念的分类学关系的纯文本问答任务上，VL模型通常优于其纯文本对应模型。通过一系列有针对性的行为和表示分析，我们发现语言模型和视觉-语言模型在分类学知识本身方面没有显著差异，但它们在表示包含分类学关系概念的问题与非分类学关系概念的问题时存在差异。这意味着分类学知识本身并未通过额外的VL训练发生实质性改变，但VL训练确实提高了在特定任务背景下部署这些知识的能力，即使该任务的呈现方式纯粹是语言性的。
英文摘要：
Does vision-and-language (VL) training change the linguistic representations of language models in meaningful ways? Most results in the literature have shown inconsistent or marginal differences, both behaviorally and representationally. In this work, we start from the hypothesis that the domain in which VL training could have a significant effect is lexical-conceptual knowledge, in particular its taxonomic organization. Through comparing minimal pairs of text-only LMs and their VL-trained counterparts, we first show that the VL models often outperform their text-only counterparts on a text-only question-answering task that requires taxonomic understanding of concepts mentioned in the questions. Using an array of targeted behavioral and representational analyses, we show that the LMs and VLMs do not differ significantly in terms of their taxonomic knowledge itself, but they differ in how they represent questions that contain concepts in a taxonomic relation vs. a non-taxonomic relation. This implies that the taxonomic knowledge itself does not change substantially through additional VL training, but VL training does improve the deployment of this knowledge in the context of a specific task, even when the presentation of the task is purely linguistic.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
