量化并实现类CLIP模型的可解释性
Quantifying and Enabling the Interpretability of CLIP-like Models
AI总结:
本研究针对CLIP内部机制不透明的问题,用TEXTSPAN算法和上下文学习拆解注意力头,以新指标量化可解释性,发现大模型可解释性更强,还推出CLIP-InterpreT分析工具助力理解。
摘要:
CLIP是最受欢迎的基础模型之一,被大量应用于众多视觉-语言任务中。然而,人们对CLIP的内部运作机制知之甚少。为填补这一空白,我们提出一项研究,对类CLIP模型的可解释性进行量化。我们在来自OpenAI和OpenCLIP的6种不同CLIP模型上开展研究,这些模型在规模、预训练数据类型和补丁大小上存在差异。我们的方法首先使用TEXTSPAN算法(文本跨度算法)和上下文学习,将单个注意力头拆解为特定属性。随后我们采用新指标评估这些注意力头的可解释程度,这些指标用于衡量头内部的属性一致性以及头之间的属性解耦性。研究结果表明,更大的CLIP模型通常比更小的同类模型更具可解释性。为进一步帮助用户理解CLIP模型的内部运作,我们推出了CLIP-InterpreT,一款专为可解释性分析设计的工具。CLIP-InterpreT提供五类分析:基于属性的最近邻搜索、逐头主题分割、对比分割、图像的逐头最近邻以及文本的逐头最近邻。
英文摘要:
CLIP is one of the most popular foundational models and is heavily used for many vision-language tasks. However, little is known about the inner workings of CLIP. To bridge this gap we propose a study to quantify the interpretability in CLIP like models. We conduct this study on six different CLIP models from OpenAI and OpenCLIP which vary by size, type of pre-training data and patch size. Our approach begins with using the TEXTSPAN algorithm and in-context learning to break down individual attention heads into specific properties. We then evaluate how easily these heads can be interpreted using new metrics which measure property consistency within heads and property disentanglement across heads. Our findings reveal that larger CLIP models are generally more interpretable than their smaller counterparts. To further assist users in understanding the inner workings of CLIP models, we introduce CLIP-InterpreT, a tool designed for interpretability analysis. CLIP-InterpreT offers five types of analyses: property-based nearest neighbor search, per-head topic segmentation, contrastive segmentation, per-head nearest neighbors of an image, and per-head nearest neighbors of text.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。