arXiv 2103.117062021-03-22stat.MLcs.LGq-fin.STstat.AP

Interpreting Deep Learning Models with Marginal Attribution by Conditioning on Quantiles

University of Hamburg（汉堡大学）；QED Actuaries & Consultants（QED精算咨询公司）；University of the Witwatersrand（威特沃特斯兰德大学）；City, University of London（伦敦城市大学）；ETH Zurich（苏黎世联邦理工学院）

M. Merz, R. Richman, T. Tsanakas, M. V. Wüthrich

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

英文摘要相关论文

英文摘要：

A vastly growing literature on explaining deep learning models has emerged. This paper contributes to that literature by introducing a global gradient-based model-agnostic method, which we call Marginal Attribution by Conditioning on Quantiles (MACQ). Our approach is based on analyzing the marginal attribution of predictions (outputs) to individual features (inputs). Specificalllly, we consider variable importance by mixing (global) output levels and, thus, explain how features marginally contribute across different regions of the prediction space. Hence, MACQ can be seen as a marginal attribution counterpart to approaches such as accumulated local effects (ALE), which study the sensitivities of outputs by perturbing inputs. Furthermore, MACQ allows us to separate marginal attribution of individual features from interaction effect, and visually illustrate the 3-way relationship between marginal attribution, output level, and feature value.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

Interpreting and improving deep-learning models with reality checks

University of California, Berkeley（加州大学伯克利分校）

Recent deep-learning models have achieved impressive predictive performance by learning complex functions of many variables, often at the cost of interpretability. This chapter covers recent work aiming to interpret models by attributing importance to features and feature groups for a single prediction. Importantly, the proposed attributions assign importance to interactions between features, in addition to features in isolation. These attributions are shown to yield insights across real-world domains, including bio-imaging, cosmology image and natural-language processing. We then show how the…

2021-08-19cs.LGstat.ML

Disentangled Attribution Curves for Interpreting Random Forests and Boosted Trees

UC Berkeley（加州大学伯克利分校）

Tree ensembles, such as random forests and AdaBoost, are ubiquitous machine learning models known for achieving strong predictive performance across a wide variety of domains. However, this strong performance comes at the cost of interpretability (i.e. users are unable to understand the relationships a trained random forest has learned and why it is making its predictions). In particular, it is challenging to understand how the contribution of a particular feature, or group of features, varies as their value changes. To address this, we introduce Disentangled Attribution Curves (DAC), a method…

2019-05-18cs.LGstat.MEstat.ML

一种用于基于Shapley值的异常分数归因的特征函数

A Characteristic Function for Shapley-Value-Based Attribution of Anomaly Scores

University of Applied Sciences and Arts Western Switzerland（瑞士西部应用科学与艺术大学）；RIKEN Center for Advanced Intelligence Project（理化学研究所先进智能项目中心）；Osaka University（大阪大学）

该研究针对异常检测中异常分数的特征归因问题，提出了一种适配Shapley值的专用特征函数，通过局部最小化近似特征缺失效果，实验验证了该归因方法的潜在效用。

2023-02-16cs.AIcs.LGstat.ML

Working women and caste in India: A study of social disadvantage using feature attribution

International Food Policy Research Institute（国际食物政策研究所）；Nanyang Technological University（南洋理工大学）

Women belonging to the socially disadvantaged caste-groups in India have historically been engaged in labour-intensive, blue-collar work. We study whether there has been any change in the ability to predict a woman's work-status and work-type based on her caste by interpreting machine learning models using feature attribution. We find that caste is now a less important determinant of work for the younger generation of women compared to the older generation. Moreover, younger women from disadvantaged castes are now more likely to be working in white-collar jobs.

2020-01-03cs.LGecon.EMstat.AP

使用innsight包解释深度神经网络

Interpreting Deep Neural Networks with the Package innsight

Leibniz Institute for Prevention Research and Epidemiology – BIPS（莱布尼茨预防研究与流行病学研究所——BIPS）；University of Bremen（不来梅大学）；University of Copenhagen（哥本哈根大学）

该论文介绍R包innsight，它提供特征归因方法解释深度神经网络预测，独立于深度学习库，支持多种模型和可视化工具。

2024-01-18cs.LGstat.ML

Learning Deep Attribution Priors Based On Prior Knowledge

University of Washington（华盛顿大学）

Feature attribution methods, which explain an individual prediction made by a model as a sum of attributions for each input feature, are an essential tool for understanding the behavior of complex deep learning models. However, ensuring that models produce meaningful explanations, rather than ones that rely on noise, is not straightforward. Exacerbating this problem is the fact that attribution methods do not provide insight as to why features are assigned their attribution values, leading to explanations that are difficult to interpret. In real-world problems we often have sets of additional …

2020-10-26cs.LGstat.ML

更多相关论文收起更多论文

Consistent Individualized Feature Attribution for Tree Ensembles

University of Washington（华盛顿大学）

Interpreting predictions from tree ensemble methods such as gradient boosting machines and random forests is important, yet feature attribution for trees is often heuristic and not individualized for each prediction. Here we show that popular feature attribution methods are inconsistent, meaning they can lower a feature's assigned importance when the true impact of that feature actually increases. This is a fundamental problem that casts doubt on any comparison between features. To address it we turn to recent applications of game theory and develop fast exact tree solutions for SHAP (SHapley …

2019-03-07cs.LGstat.ML

逐层归因？可解释性中的元游戏

The Metagame of Interpretability and Meta-Attributions

University of Warsaw（华沙大学）；Centre for Credible AI, Warsaw University of Technology（可信AI中心，华沙技术大学）；Bielefeld University（比勒菲尔德大学）；LMU Munich, MCML（慕尼黑大学LMU，MCML）

本文提出元游戏框架，通过将归因方法视为合作博弈计算Shapley值，研究模型解释的第二阶交互效应，揭示归因的层级分解及其在多领域可解释性应用中的价值。

2026-10-08cs.AIcs.LGstat.ML

RoSHAP：一种分布框架和稳健度量用于稳定的特征归因

RoSHAP: A Distributional Framework and Robust Metric for Stable Feature Attribution

Department of Statistics（统计学系）；Virginia Tech（弗吉尼亚理工大学）；Transportation Institute（交通运输研究所）；Department of Computer Science（计算机科学系）

本文提出RoSHAP框架和稳健度量，通过Bootstrap重采样和核密度估计建模特征归因分布，提升特征归因的稳定性与可解释性。

2026-05-14cs.LGstat.ML

贪婪PIG：自适应积分梯度

Greedy PIG: Adaptive Integrated Gradients

Google Research（谷歌研究院）

本文提出贪婪PIG，一种基于子集选择的自适应积分梯度方法，统一特征归因与选择，在图像、图及表格任务上显著提升归因性能。

2023-11-10cs.AIcs.LGstat.ML

A Causal Lens for Peeking into Black Box Predictive Models: Predictive Model Interpretation via Causal Attribution

The Pennsylvania State University（宾夕法尼亚州立大学）

With the increasing adoption of predictive models trained using machine learning across a wide range of high-stakes applications, e.g., health care, security, criminal justice, finance, and education, there is a growing need for effective techniques for explaining such models and their predictions. We aim to address this problem in settings where the predictive model is a black box; That is, we can only observe the response of the model to various inputs, but have no knowledge about the internal structure of the predictive model, its parameters, the objective function, and the algorithm used t…

2020-08-01cs.AIcs.LGstat.ML