arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.19271cs.AIcs.LG
收藏

基于概念的可解释人工智能:指标与基准

Concept-Based Explainable Artificial Intelligence: Metrics and Benchmarks

University of Southampton(南安普顿大学);Computer Science(计算机科学学院)

Halil Ibrahim Aysel, Xiaohao Cai, Adam Prugel-Bennett

AI总结:

针对概念瓶颈模型中概念在特征空间对齐假设缺乏验证的问题,提出了概念全局重要性、存在性和定位三个指标及可视化技术,揭示了事后CBMs在概念对齐上的严重局限。

摘要:

基于概念的解释方法,例如概念瓶颈模型(CBMs),旨在通过将机器学习模型的决策与人类可理解的概念联系起来以提高模型的可解释性,其关键假设是这些概念能够被准确归因到网络的特征空间中。然而,这一基础性假设尚未得到严格验证,主要因为该领域缺乏标准化的指标和基准来评估此类概念的存在性和空间对齐情况。为了解决这一问题,我们提出了三个指标:概念全局重要性指标、概念存在性指标和概念定位指标,包括一种可视化概念激活的技术,即概念激活映射。我们对事后 CBMs 进行了基准测试,以说明它们的能力与面临的挑战。通过定性和定量实验,我们证明在许多情况下,即使是由事后 CBMs 确定的最重要的概念也并未出现在输入图像中;此外,当这些概念确实存在时,其显著性图也未能与预期区域对齐,要么在整个对象上激活,要么错误识别了相关的特定概念区域。我们分析了这些局限性的根本原因,例如概念的自然相关性。我们的研究结果强调,需要更加谨慎地应用基于概念的解释技术,尤其是在空间可解释性至关重要的场景中。

英文摘要:

Concept-based explanation methods, such as concept bottleneck models (CBMs), aim to improve the interpretability of machine learning models by linking their decisions to human-understandable concepts, under the critical assumption that such concepts can be accurately attributed to the network's feature space. However, this foundational assumption has not been rigorously validated, mainly because the field lacks standardised metrics and benchmarks to assess the existence and spatial alignment of such concepts. To address this, we propose three metrics: the concept global importance metric, the concept existence metric, and the concept location metric, including a technique for visualising concept activations, i.e., concept activation mapping. We benchmark post-hoc CBMs to illustrate their capabilities and challenges. Through qualitative and quantitative experiments, we demonstrate that, in many cases, even the most important concepts determined by post-hoc CBMs are not present in input images; moreover, when they are present, their saliency maps fail to align with the expected regions by either activating across an entire object or misidentifying relevant concept-specific regions. We analyse the root causes of these limitations, such as the natural correlation of concepts. Our findings underscore the need for more careful application of concept-based explanation techniques especially in settings where spatial interpretability is critical.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑