arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.08499cs.LGcs.AI
收藏

XAISuite框架及解释系统不一致性的影响

The XAISuite framework and the implications of explanatory system dissonance

Adrian C. Wilcox High School(阿德里安·C·威尔科克斯高中);UC Santa Cruz(加州大学圣克鲁兹分校)

Shreyan Mitra, Leilani Gilpin

AI总结:

本文通过比较SHAP和LIME在多种模型与数据集上的重要性分数,发现重要性大小不决定解释一致性且解释相似性无法预测准确率,并提出XAISuite开源库与通用解释优化框架。

摘要:

解释系统使机器学习模型更加透明。然而,它们往往并不一致。为了量化和分离可能导致这种差异的场景,本文基于14个机器学习模型(7个回归模型和7个分类模型)和4个表格数据集(2个回归数据集和2个分类数据集),通过比较SHAP和LIME各自重要性分数的相关性,对这两种解释系统进行了对比。我们得出了两个新发现。首先,重要性的大小在解释一致性中并不显著。对于最重要的特征,SHAP和LIME重要性分数之间的相关性,与所有特征上平均的SHAP和LIME重要性分数之间的相关性相比,可能更具变异性,也可能并非如此。其次,SHAP和LIME重要性分数之间的相似性无法预测模型准确率。在研究过程中,我们构建了一个开源库XAISuite,统一了训练和解释模型的流程。最后,本文贡献了一个通用框架,以更好地解释机器学习模型并优化其性能。

英文摘要:

Explanatory systems make machine learning models more transparent. However, they are often inconsistent. In order to quantify and isolate possible scenarios leading to this discrepancy, this paper compares two explanatory systems, SHAP and LIME, based on the correlation of their respective importance scores using 14 machine learning models (7 regression and 7 classification) and 4 tabular datasets (2 regression and 2 classification). We make two novel findings. Firstly, the magnitude of importance is not significant in explanation consistency. The correlations between SHAP and LIME importance scores for the most important features may or may not be more variable than the correlation between SHAP and LIME importance scores averaged across all features. Secondly, the similarity between SHAP and LIME importance scores cannot predict model accuracy. In the process of our research, we construct an open-source library, XAISuite, that unifies the process of training and explaining models. Finally, this paper contributes a generalized framework to better explain machine learning models and optimize their performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑