arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2206.09677cs.LGcs.AI
收藏

GraphFramEx:面向图神经网络可解释性方法的系统评估框架

GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks

ETH Zürich(苏黎世联邦理工学院);Yale University(耶鲁大学);eBay China(eBay中国)

Kenza Amara, Rex Ying, Zitao Zhang, Zhihao Han, Yinan Shan, Ulrik Brandes, Sebastian Schemm, Ce Zhang

AI总结:

本文提出首个系统评估GNN可解释性方法的框架GraphFramEx,结合充分性与必要性度量,比较多种方法,发现浅层方法在简单基准上最优,而梯度方法在复杂图上最佳,并应用于eBay欺诈解释案例。

摘要:

作为当今最流行的机器学习模型之一,图神经网络(GNNs)近来引起了强烈关注,其可解释性同样备受关注。用户越来越希望更好地理解GNN模型及其输出结果。然而,目前GNN可解释性的评估框架往往依赖于少量不充分的合成数据集,由于问题实例缺乏复杂性,导致结论适用范围有限。随着GNN模型被部署到更多关键任务应用中,我们迫切需要一种通用的GNN可解释性方法评估协议。在本文中,我们提出了据我们所知首个针对GNN可解释性的系统评估框架,考虑了三种不同“用户需求”下的可解释性。我们提出了一种独特的指标,该指标结合了保真度度量,并根据解释是否充分或必要对其质量进行分类。我们将研究范围限定在节点分类任务,并比较了GNN输入级可解释性领域中最具代表性的技术。对于不充分但广泛使用的合成基准,令人惊讶的是,诸如个性化PageRank之类的浅层技术在最短计算时间内取得了最佳性能。但当图结构更复杂且节点具有有意义的特征时,基于梯度的方法根据我们的评估标准表现最佳。然而,没有任何方法在所有评估维度上占据主导地位,总是存在权衡。我们进一步将我们的评估协议应用于eBay交易图上的欺诈解释案例研究,以反映生产环境。

英文摘要:

As one of the most popular machine learning models today, graph neural networks (GNNs) have attracted intense interest recently, and so does their explainability. Users are increasingly interested in a better understanding of GNN models and their outcomes. Unfortunately, today's evaluation frameworks for GNN explainability often rely on few inadequate synthetic datasets, leading to conclusions of limited scope due to a lack of complexity in the problem instances. As GNN models are deployed to more mission-critical applications, we are in dire need for a common evaluation protocol of explainability methods of GNNs. In this paper, we propose, to our best knowledge, the first systematic evaluation framework for GNN explainability, considering explainability on three different "user needs". We propose a unique metric that combines the fidelity measures and classifies explanations based on their quality of being sufficient or necessary. We scope ourselves to node classification tasks and compare the most representative techniques in the field of input-level explainability for GNNs. For the inadequate but widely used synthetic benchmarks, surprisingly shallow techniques such as personalized PageRank have the best performance for a minimum computation time. But when the graph structure is more complex and nodes have meaningful features, gradient-based methods are the best according to our evaluation criteria. However, none dominates the others on all evaluation dimensions and there is always a trade-off. We further apply our evaluation protocol in a case study for frauds explanation on eBay transaction graphs to reflect the production environment.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑