arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2302.12893cs.LGcs.AI
收藏

不要被愚弄:解释方法中的标签泄漏及其定量评价的重要性

Don't be fooled: label leakage in explanation methods and the importance of their quantitative evaluation

Grossman School of Medicine(格罗斯曼医学院);Courant Institute(库朗研究所);Center for Data Science(数据科学中心);New York University(纽约大学)

Neil Jethani, Adriel Saporta, Rajesh Ranganath

摘要:

特征属性方法识别输入的哪些特征对模型的输出影响最大。大多数广泛使用的特征属性方法(如SHAP、LIME和Grad-CAM)都是“类依赖”方法,因为它们生成的特征属性向量是类的函数。在这项工作中,我们证明了类相关的方法可以“泄漏”有关所选类的信息,使该类看起来比实际情况更有可能。因此,最终用户在解释依赖于类的方法生成的解释时会冒得出错误结论的风险。相比之下,我们引入了“分布感知”的方法,这些方法有利于解释保持标签的分布接近其分布,因为输入的所有特征都是如此。我们介绍SHAP-KL和FastSHAP-KL,这两种基线分布感知方法可以计算Shapley值。最后,我们对不同高维数据类型(图像、生物信号和文本)的三个临床数据集进行了七种类依赖和三种分布感知方法的综合评估。

英文摘要:

Feature attribution methods identify which features of an input most influence a model's output. Most widely-used feature attribution methods (such as SHAP, LIME, and Grad-CAM) are "class-dependent" methods in that they generate a feature attribution vector as a function of class. In this work, we demonstrate that class-dependent methods can "leak" information about the selected class, making that class appear more likely than it is. Thus, an end user runs the risk of drawing false conclusions when interpreting an explanation generated by a class-dependent method. In contrast, we introduce "distribution-aware" methods, which favor explanations that keep the label's distribution close to its distribution given all features of the input. We introduce SHAP-KL and FastSHAP-KL, two baseline distribution-aware methods that compute Shapley values. Finally, we perform a comprehensive evaluation of seven class-dependent and three distribution-aware methods on three clinical datasets of different high-dimensional data types: images, biosignals, and text.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑