HYDRA：用于解释深度神经网络的超梯度数据相关性分析
HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks
AI总结：
本文提出 HYDRA，通过展开超梯度评估训练数据对测试数据点在整个训练轨迹中的贡献，移除 Hessian 加速计算并证明误差有界，在估计数据贡献和检测噪声标签上优于影响函数。
摘要：
深度神经网络（DNN）的行为极难被人类解释。本文提出超梯度数据相关性分析（HYDRA），将 DNN 的预测解释为其训练数据的影响。现有方法通常在最终模型参数附近估计数据贡献，忽略了训练数据如何塑造优化轨迹。通过展开测试损失关于训练数据权重的超梯度，HYDRA 评估了在整个训练轨迹中训练数据对测试数据点的贡献。为加速计算，我们移除了计算中的 Hessian，并证明在适度条件下近似误差是有界的。实证结果证实了这一理论主张，表明误差确实很小。此外，我们定量证明 HYDRA 在准确估计数据贡献和检测噪声数据标签方面优于影响函数。源代码见 https://github.com/cyyever/aaai_hydra_8686。
英文摘要：
The behaviors of deep neural networks (DNNs) are notoriously resistant to human interpretations. In this paper, we propose Hypergradient Data Relevance Analysis, or HYDRA, which interprets the predictions made by DNNs as effects of their training data. Existing approaches generally estimate data contributions around the final model parameters and ignore how the training data shape the optimization trajectory. By unrolling the hypergradient of test loss w.r.t. the weights of training data, HYDRA assesses the contribution of training data toward test data points throughout the training trajectory. In order to accelerate computation, we remove the Hessian from the calculation and prove that, under moderate conditions, the approximation error is bounded. Corroborating this theoretical claim, empirical results indicate the error is indeed small. In addition, we quantitatively demonstrate that HYDRA outperforms influence functions in accurately estimating data contribution and detecting noisy data labels. The source code is available at https://github.com/cyyever/aaai_hydra_8686.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
