arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.19669cs.LGcs.AIcs.CLcs.CV
收藏

分析语言与视觉在有限数据学习中的作用

Analyzing the Roles of Language and Vision in Learning from Limited Data

Princeton University(普林斯顿大学)

Allison Chen, Ilia Sucholutsky, Olga Russakovsky, Thomas L. Griffiths

AI总结:

本文通过消融视觉-语言模型的认知组件,研究语言与视觉在有限数据新任务学习中的作用,发现语言模型可借助先验知识与推理恢复 VLM 的大部分性能。

摘要:

语言是否有助于理解视觉世界?真正看见世界,而不是通过文字描述来了解世界,究竟有多重要?这些关于智能本质的基础问题一直难以回答,因为我们只有一个智能系统的例子——人类——而且很难接触到将语言或视觉单独隔离出来的案例。然而,人工智能研究人员开发的复杂视觉-语言模型(Vision-Language Models,VLMs)为我们探索语言和视觉在认识世界过程中所做的贡献提供了新的机会。我们对这些模型认知架构中的组件进行消融,以确定它们在从有限数据中学习新任务时的贡献。我们发现,一个利用所有组件的语言模型尽管缺乏视觉输入,却能恢复 VLM 的大部分性能;语言似乎通过提供获取先验知识和推理的途径来实现这一点。

英文摘要:

Does language help make sense of the visual world? How important is it to actually see the world rather than having it described with words? These basic questions about the nature of intelligence have been difficult to answer because we only had one example of an intelligent system -- humans -- and limited access to cases that isolated language or vision. However, the development of sophisticated Vision-Language Models (VLMs) by artificial intelligence researchers offers us new opportunities to explore the contributions that language and vision make to learning about the world. We ablate components from the cognitive architecture of these models to identify their contributions to learning new tasks from limited data. We find that a language model leveraging all components recovers a majority of a VLM's performance, despite its lack of visual input, and that language seems to allow this by providing access to prior knowledge and reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑