arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2303.00652cs.LGcs.AI
收藏

寻找合适的XAI方法——气候科学中可解释人工智能方法的评估与排序指南

Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science

Technische Universität Berlin（柏林工业大学）；ATB（莱布尼茨农业工程与生物经济研究所）；University of Leipzig（莱比锡大学）；University of Reading（雷丁大学）；BIFOLD – Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所（BIFOLD））；UiT the Arctic University of Norway（挪威北极大学）；University of Potsdam（波茨坦大学）

Philine Bommer, Marlene Kretschmer, Anna Hedström, Dilyara Bareeva, Marina M. -C. Höhne

备注：
19 pages, 10 figure, accepted at AIES journal by AMS
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文在气候科学背景下引入XAI方法评估，通过分析鲁棒性、忠实性等属性，比较多种XAI方法在不同网络架构上的表现，为气候研究人员选择合适的解释方法提供指南。

摘要：

可解释人工智能（XAI）方法为机器学习算法的预测提供了透明度。目前已有多种不同的方法，并且它们已被应用于气候科学中。然而，由于通常缺乏真实的基准解释，使得对这些方法的评估和比较变得复杂，进而阻碍了XAI方法的选择。因此，在这项工作中，我们在气候背景下引入了XAI评估，并讨论了不同的期望解释属性，即鲁棒性、忠实性、随机性、复杂性和局部性。为此，我们选择了一项先前的工作作为案例研究，其中预测了年平均温度图的年代。在训练了多层感知机（MLP）和卷积神经网络（CNN）之后，我们应用了多种XAI方法，并针对每种属性计算了它们相对于随机均匀解释的技能得分。我们发现，独立于网络架构，XAI方法Integrated Gradients、层级相关性传播和输入乘以梯度表现出相当的鲁棒性、忠实性和复杂性，但牺牲了随机性表现。敏感性方法——梯度、SmoothGrad、NoiseGrad和FusionGrad，匹配了鲁棒性技能，但为了随机性技能牺牲了忠实性和复杂性。我们发现不同XAI方法在鲁棒性、复杂性和局部性技能方面存在依赖于架构的性能差异，突出了针对研究任务进行特定评估的必要性。总体而言，我们的工作提供了气候科学背景下不同评估属性的概述，并展示了如何比较和基准测试不同的解释方法，根据其优缺点评估它们对当前特定研究问题的适用性。通过这种方式，我们旨在支持气候研究人员选择合适的XAI方法。

英文摘要：

Explainable artificial intelligence (XAI) methods shed light on the predictions of machine learning algorithms. Several different approaches exist and have already been applied in climate science. However, usually missing ground truth explanations complicate their evaluation and comparison, subsequently impeding the choice of the XAI method. Therefore, in this work, we introduce XAI evaluation in the climate context and discuss different desired explanation properties, namely robustness, faithfulness, randomization, complexity, and localization. To this end, we chose previous work as a case study where the decade of annual-mean temperature maps is predicted. After training both a multi-layer perceptron (MLP) and a convolutional neural network (CNN), multiple XAI methods are applied and their skill scores in reference to a random uniform explanation are calculated for each property. Independent of the network, we find that XAI methods Integrated Gradients, layer-wise relevance propagation, and input times gradients exhibit considerable robustness, faithfulness, and complexity while sacrificing randomization performance. Sensitivity methods -- gradient, SmoothGrad, NoiseGrad, and FusionGrad, match the robustness skill but sacrifice faithfulness and complexity for randomization skill. We find architecture-dependent performance differences regarding robustness, complexity and localization skills of different XAI methods, highlighting the necessity for research task-specific evaluation. Overall, our work offers an overview of different evaluation properties in the climate science context and shows how to compare and benchmark different explanation methods, assessing their suitability based on strengths and weaknesses, for the specific research problem at hand. By that, we aim to support climate researchers in the selection of a suitable XAI method.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑