模型修正的检验
Debugging Tests for Model Explanations
摘要：
我们调查是否事后模型的解释是有效的诊断模型错误-模型调试。为了应对解释模型预测的挑战，已经提出了大量的解释方法。尽管使用越来越多，但尚不清楚它们是否有效。首先，我们根据其来源将\textit{bugs}分类为：~ \textit{data, model, and test-time}污染缺陷。对于几种解释方法，我们评估它们的能力：检测虚假的相关性伪影（数据污染），诊断错误标记的训练样本（数据污染），区分（部分）重新初始化的模型和训练的模型（模型污染），以及检测分布外的输入（测试时污染）。我们发现，测试的方法能够诊断一个虚假的背景错误，但不能最终确定错误标记的训练样本。此外，一类修改反向传播算法的方法对于深度网络的高层参数是不变的;因此，对于诊断模型污染是无效的。我们补充我们的分析与人类受试者的研究，发现受试者无法识别有缺陷的模型使用属性，而是主要依赖于模型预测。总之，我们的研究结果提供了指导的从业者和研究人员转向解释模型调试的工具。
英文摘要：
We investigate whether post-hoc model explanations are effective for diagnosing model errors--model debugging. In response to the challenge of explaining a model's prediction, a vast array of explanation methods have been proposed. Despite increasing use, it is unclear if they are effective. To start, we categorize \textit{bugs}, based on their source, into:~\textit{data, model, and test-time} contamination bugs. For several explanation methods, we assess their ability to: detect spurious correlation artifacts (data contamination), diagnose mislabeled training examples (data contamination), differentiate between a (partially) re-initialized model and a trained one (model contamination), and detect out-of-distribution inputs (test-time contamination). We find that the methods tested are able to diagnose a spurious background bug, but not conclusively identify mislabeled training examples. In addition, a class of methods, that modify the back-propagation algorithm are invariant to the higher layer parameters of a deep network; hence, ineffective for diagnosing model contamination. We complement our analysis with a human subject study, and find that subjects fail to identify defective models using attributions, but instead rely, primarily, on model predictions. Taken together, our results provide guidance for practitioners and researchers turning to explanations as tools for model debugging.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
