arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.08652cs.LGcs.AIcs.HC
收藏

TextCAVs:使用文本调试视觉模型

TextCAVs: Debugging vision models using text

Institute of Biomedical Engineering, University of Oxford(牛津大学生物医学工程研究所);OATML, Department of Computer Science, University of Oxford(牛津大学计算机科学系OATML)

Angus Nicolson, Yarin Gal, J. Alison Noble

AI总结:

针对传统概念激活向量方法需标注图像成本高的问题,提出TextCAVs方法,借助CLIP等视觉-语言模型仅用文本描述生成CAVs,在MIMIC-CXR和ImageNet上验证有效,可用于调试深度学习视觉模型。

摘要:

基于概念的可解释性方法是一种流行的深度学习模型解释形式,它以人类可理解的高层概念的形式提供解释。这类方法通常使用包含概念示例的探测数据集来找到概念激活向量(CAVs),这需要为这些概念准备标注数据——在医学领域这是一项成本高昂的任务。我们提出了TextCAVs:一种利用CLIP等视觉-语言模型生成CAVs的新方法,仅使用概念的文本描述即可生成解释,无需图像样本。这种概念测试成本的降低使得大量概念可以被测试,用户还可以与模型交互,想到新的想法就能立即测试,而无需因图像收集和标注产生延迟。在初步实验结果中,我们证明TextCAVs在胸部X光数据集(MIMIC-CXR)和自然图像(ImageNet)上都能生成合理的解释,且这些解释可用于调试基于深度学习的模型。

英文摘要:

Concept-based interpretability methods are a popular form of explanation for deep learning models which provide explanations in the form of high-level human interpretable concepts. These methods typically find concept activation vectors (CAVs) using a probe dataset of concept examples. This requires labelled data for these concepts -- an expensive task in the medical domain. We introduce TextCAVs: a novel method which creates CAVs using vision-language models such as CLIP, allowing for explanations to be created solely using text descriptions of the concept, as opposed to image exemplars. This reduced cost in testing concepts allows for many concepts to be tested and for users to interact with the model, testing new ideas as they are thought of, rather than a delay caused by image collection and annotation. In early experimental results, we demonstrate that TextCAVs produces reasonable explanations for a chest x-ray dataset (MIMIC-CXR) and natural images (ImageNet), and that these explanations can be used to debug deep learning-based models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑