arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.08137cs.LG
收藏

归一化AOPC:修复特征归因可解释性中误导性的忠实度指标

Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability

Corti;University of Copenhagen(哥本哈根大学);IT University of Copenhagen(哥本哈根技术大学);LUT University(拉普兰大学)

Joakim Edin, Andreas Geert Motzfeldt, Casper L. Christensen, Tuukka Ruotsalo, Lars Maaløe, Maria Maistro

AI总结:

针对AOPC指标在跨模型比较中的误导性问题,提出归一化AOPC(NAOPC),实现一致的忠实度评估,并质疑早期研究结论,提供更稳健的框架。

摘要:

深度神经网络预测因难以解释而闻名。特征归因方法旨在通过识别每个输入特征的贡献来解释这些预测。忠实度通常使用扰动曲线下面积(AOPC)来评估,反映特征归因在描述深度神经网络内部机制方面的准确性。然而,许多研究依赖AOPC来比较不同模型的忠实度,我们表明这可能导致关于模型忠实度的错误结论。具体来说,我们发现AOPC对模型的变化敏感,导致跨模型比较不可靠。此外,在没有知道模型特定的下限和上限的情况下,AOPC分数难以孤立解释。为解决这些问题,我们提出了一种归一化方法,即归一化AOPC(NAOPC),实现一致的跨模型评估和更有意义的单个分数解释。我们的实验表明,这种归一化可以彻底改变AOPC结果,质疑早期研究的结论,并为评估特征归因忠实度提供更稳健的框架。我们的代码可在https://github.com/JoakimEdin/naopc获取。

英文摘要:

Deep neural network predictions are notoriously difficult to interpret. Feature attribution methods aim to explain these predictions by identifying the contribution of each input feature. Faithfulness, often evaluated using the area over the perturbation curve (AOPC), reflects feature attributions' accuracy in describing the internal mechanisms of deep neural networks. However, many studies rely on AOPC to compare faithfulness across different models, which we show can lead to false conclusions about models' faithfulness. Specifically, we find that AOPC is sensitive to variations in the model, resulting in unreliable cross-model comparisons. Moreover, AOPC scores are difficult to interpret in isolation without knowing the model-specific lower and upper limits. To address these issues, we propose a normalization approach, Normalized AOPC (NAOPC), enabling consistent cross-model evaluations and more meaningful interpretation of individual scores. Our experiments demonstrate that this normalization can radically change AOPC results, questioning the conclusions of earlier studies and offering a more robust framework for assessing feature attribution faithfulness. Our code is available at https://github.com/JoakimEdin/naopc.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑