arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09238cs.LG
收藏

特征显著性 - 而不是任务信息性 - 决定了机器学习模型解释

Feature salience - not task-informativeness - drives machine learning model explanations

Benedict Clark, Marta Oliveira, Rick Wilming, Stefan Haufe

AI总结:

研究发现,机器学习模型解释的重要性主要由测试时图像结构的显著性驱动,而非统计关联。

摘要:

可解释人工智能(XAI)承诺提供对机器学习模型决策过程的洞察,其中一项目标是识别失败情况,如捷径学习。这一承诺依赖于该领域的一个假设,即XAI标记为重要的输入特征必须包含关于目标变量的信息。然而,不清楚信息性是否确实是重要性分配的主要驱动因素,或者是否其他数据属性,如统计抑制、测试时的新颖性或高特征显著性,实质上也有所贡献。为了澄清这一点,我们训练了深度学习模型,以三种二元图像分类任务的变体进行训练,在这些任务中,半透明水印要么不存在,要么作为类依赖的干扰因素,要么代表类无关的噪声。五种流行的重要性归因方法的结果显示,在水印区域(RIW)的相对重要性显著升高,无论训练设置如何(R²≥.45)。相比之下,水印是否存在是否是类依赖的,仅对RIW有微小影响(R²≤.03),尽管对模型性能和泛化能力有明显影响。XAI方法表现出与模型无关的边缘检测滤波器相似的行为,并且当明亮图像强度由较小而不是较大的特征值编码时,它们对水印的重要性归因要少得多。这些结果表明,重要性归因主要由测试时图像结构的显著性驱动,而不是机器学习模型学习的统计关联。先前显示成功XAI应用的研究应重新评估,考虑到可能的伪相关性特征显著性和信息性,以及使用特征归因方法作为构建块的工作流程应受到审查。

英文摘要:

Explainable AI (XAI) promises to provide insight into machine learning models' decision processes, where one goal is to identify failures such as shortcut learning. This promise relies on the field's assumption that input features marked as important by an XAI must contain information about the target variable. However, it is unclear whether informativeness is indeed the main driver of importance attribution in practice, or if other data properties such as statistical suppression, novelty at test-time, or high feature salience substantially contribute. To clarify this, we trained deep learning models on three variants of a binary image classification task, in which translucent watermarks are either absent, act as class-dependent confounds, or represent class-independent noise. Results for five popular attribution methods show substantially elevated relative importance in watermarked areas (RIW) for all models regardless of the training setting ($R^2 \geq .45$). By contrast, whether the presence of watermarks is class-dependent or not only has a marginal effect on RIW ($R^2 \leq .03$), despite a clear impact impact on model performance and generalisation ability. XAI methods show similar behaviour to model-agnostic edge detection filters and attribute substantially less importance to watermarks when bright image intensities are encoded by smaller instead of larger feature values. These results indicate that importance attribution is most strongly driven by the salience of image structures at test time rather than statistical associations learned by machine learning models. Previous studies demonstrating successful XAI application should be reevaluated with respect to a possibly spurious concurrency of feature salience and informativeness, and workflows using feature attribution methods as building blocks should be scrutinised.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑