面向大型多模态模型的基于概念的可解释性框架
A Concept-Based Explainability Framework for Large Multimodal Models
AI总结：
针对大型多模态模型内部表示难解释的问题，提出基于字典学习的token表示概念提取框架，获得兼具视觉与文本语义基础的多模态概念，并验证其可解释性、解耦性与 grounding 质量。
摘要：
大型多模态模型（LMM）结合单模态编码器和大型语言模型（LLM）来执行多模态任务。尽管这些模型的可解释性近期取得了进展，但理解LMM的内部表示在很大程度上仍是一个谜。本文提出了一种用于解释LMM的新颖框架。我们提出一种基于字典学习的方法，并将其应用于token表示。学习到的字典元素对应于我们提出的概念。我们表明，这些概念在视觉和文本两方面都具有良好的语义基础，因此将其称为“多模态概念”。我们对所学概念的结果进行了定性和定量评估，表明提取出的多模态概念有助于解释测试样本的表示。最后，我们评估了不同概念之间的解耦，以及概念在视觉和文本上的基础质量。我们的代码可在https://github.com/mshukor/xl-vlms公开获取。
英文摘要：
Large multimodal models (LMMs) combine unimodal encoders and large language models (LLMs) to perform multimodal tasks. Despite recent advancements towards the interpretability of these models, understanding internal representations of LMMs remains largely a mystery. In this paper, we present a novel framework for the interpretation of LMMs. We propose a dictionary learning based approach, applied to the representation of tokens. The elements of the learned dictionary correspond to our proposed concepts. We show that these concepts are well semantically grounded in both vision and text. Thus we refer to these as ``multi-modal concepts''. We qualitatively and quantitatively evaluate the results of the learnt concepts. We show that the extracted multimodal concepts are useful to interpret representations of test samples. Finally, we evaluate the disentanglement between different concepts and the quality of grounding concepts visually and textually. Our code is publicly available at https://github.com/mshukor/xl-vlms
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
