arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.07149cs.CVcs.LG
收藏

迈向视觉语言模型中视觉信息处理的可解释性

Towards Interpreting Visual Information Processing in Vision-Language Models

Nanyang Technological University(南洋理工大学);University of Oxford(牛津大学);Tel Aviv University(特拉维夫大学);MILA(蒙特利尔人工智能研究院);ERA-Krueger AI Safety Lab(ERA-Krueger人工智能安全实验室)

Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez

AI总结:

本文研究LLaVA视觉语言模型中视觉token的处理机制,发现移除特定token使识别率降超70%,且模型在最后token位置提取信息,为多模态系统可解释性提供见解。

摘要:

视觉语言模型(VLMs)是处理和理解文本与图像的强大工具。我们研究了LLaVA(一个突出的VLM)中语言模型组件对视觉token的处理。我们的方法侧重于分析物体信息的定位、视觉token表示在各层中的演变,以及整合视觉信息进行预测的机制。通过消融研究,我们证明当移除特定物体的token时,物体识别准确率下降超过70%。我们观察到,视觉token表示在各层的词汇空间中变得越来越可解释,这表明其与图像内容对应的文本token实现了对齐。最后,我们发现模型在最后一个token位置从这些精炼的表示中提取物体信息进行预测,这反映了纯文本语言模型在事实关联任务中的处理过程。这些发现为VLMs如何处理和整合视觉信息提供了关键见解,弥合了我们对语言和视觉模型理解之间的差距,并为更具可解释性和可控性的多模态系统铺平了道路。

英文摘要:

Vision-Language Models (VLMs) are powerful tools for processing and understanding text and images. We study the processing of visual tokens in the language model component of LLaVA, a prominent VLM. Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions. Through ablation studies, we demonstrated that object identification accuracy drops by over 70\% when object-specific tokens are removed. We observed that visual token representations become increasingly interpretable in the vocabulary space across layers, suggesting an alignment with textual tokens corresponding to image content. Finally, we found that the model extracts object information from these refined representations at the last token position for prediction, mirroring the process in text-only language models for factual association tasks. These findings provide crucial insights into how VLMs process and integrate visual information, bridging the gap between our understanding of language and vision models, and paving the way for more interpretable and controllable multimodal systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑