Towards Interpreting Visual Information Processing in Vision-Language Models
Nanyang Technological University(南洋理工大学);University of Oxford(牛津大学);Tel Aviv University(特拉维夫大学);MILA(蒙特利尔人工智能研究院);ERA-Krueger AI Safety Lab(ERA-Krueger人工智能安全实验室)
Clement Neo, Luke Ong, Philip Torr, Mor Geva, David Krueger, Fazl Barez
Vision-Language Models (VLMs) are powerful tools for processing and understanding text and images. We study the processing of visual tokens in the language model component of LLaVA, a prominent VLM. Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions. Through ablation studies, we demonstrated that object identification accuracy drops by over 70\% when object-specific tokens are removed. We observed that visual token representations become increasingly interpretable in the vocabulary space across layers, suggesting an alignment with textual tokens corresponding to image content. Finally, we found that the model extracts object information from these refined representations at the last token position for prediction, mirroring the process in text-only language models for factual association tasks. These findings provide crucial insights into how VLMs process and integrate visual information, bridging the gap between our understanding of language and vision models, and paving the way for more interpretable and controllable multimodal systems.
What do VLM-Based Vision-Language Navigation Models Rely on: Interpreting and Steering Policy Behavior
University of Technology Nuremberg(纽伦堡工业大学);ETH Zürich(苏黎世联邦理工学院);Inria, Ecole Normale Supérieure, CNRS, PSL Research University(法国国家信息与自动化研究所,巴黎高等师范学校,法国国家科学研究中心,巴黎文理研究大学)
NumerosityVLM: A Cognitively Inspired Benchmark for Interpreting Numerosity Representations in Vision-Language Models
The University of Manchester(曼彻斯特大学);University of Michigan(密歇根大学);Sheffield Hallam University(谢菲尔德哈勒姆大学);Tufts University(塔夫茨大学);ENSTA, Institut Polytechnique de Paris(巴黎综合理工学院国立高等先进技术学校)
From Behavioral Performance to Internal Competence: Interpreting Vision-Language Models with VLM-Lens
University of Waterloo(滑铁卢大学);Vector Institute(向量研究所);University of Michigan(密歇根大学);Stony Brook University(石溪大学);McGill University(麦吉尔大学);Dartmouth College(达特茅斯学院)
The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision
MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);IIIS, Tsinghua University(清华大学交叉信息研究院);MIT-IBM Watson AI Lab(麻省理工学院-IBM沃森人工智能实验室);Deepmind(深度思考);MIT BCS(麻省理工学院脑与认知科学系);CBMM(脑、心智与机器中心)
Does AI See like Art Historians? Interpreting How Vision Language Models Recognize Artistic Style
Columbia University, Department of Computer Science(哥伦比亚大学计算机科学系);Columbia University, Department of Art History & Archaeology(哥伦比亚大学艺术史与考古系);University of Texas at Austin(德克萨斯大学奥斯汀分校);UNC Chapel Hill(北卡罗来纳大学教堂山分校)
Interpreting Object-level Foundation Models via Visual Precision Search
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院);School of Computing, NUS(新加坡国立大学计算机学院);Huawei Technologies Co., Ltd.(华为技术有限公司);College of Computer, NUDT(国防科技大学计算机学院);Key Lab. of Edu. Inf. for Nationalities (YNNU), Ministry of Education(云南民族大学民族教育信息化教育部重点实验室);School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区网络空间安全学院)
WSI-VQA: Interpreting Whole Slide Images by Generative Visual Question Answering
Zhejiang University(浙江大学);Research Center for Industries of the Future, Westlake University(西湖大学未来产业研究中心);School of Engineering, Westlake University(西湖大学工程学院)
Interpreting the linear structure of vision-language model embedding spaces
Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University(哈佛大学自然与人工智能研究所);Department of Computer Science, Harvard University(哈佛大学计算机科学系)
SKLCCSE, School of Artificial Intelligence, Beihang University, Beijing, China(北京航空航天大学人工智能学院);Hangzhou International Innovation Institute, Beihang University, Hangzhou, China(北京航空航天大学杭州国际创新研究院)
RVSD: Retrieval Vision Sparse Decoding for Mitigating Visual Hallucinations in Large Vision-Language Models
School of Automation, Guangdong University of Technology(广东工业大学自动化学院);City University of Hong Kong(香港城市大学);The Second Affiliated Hospital of Guangzhou University of Chinese Medicine(广州中医药大学第二附属医院)
Generative Visual Communication in the Era of Vision-Language Models
Raymond and Beverly Sackler Faculty of Exact Sciences(雷蒙德和贝弗利·萨克勒精密科学学院);Blavatnik School of Computer Science(布拉瓦尼克计算机科学学院);School of Computer Science at Tel Aviv University(特拉维夫大学计算机科学学院)