通过影响矩阵估计实现大规模数据归因

中文摘要 AI 辅助

数据归因旨在量化单个训练样本如何影响模型预测，并支撑数据估值、机器遗忘和模型可解释性等问题。尽管已有大量研究工作，但由于神经网络的非凸性，计算上可扩展的方法往往难以准确预测移除训练数据所带来的影响。为克服这一挑战，基于元梯度的方法（如MAGIC（Ilyas和Engstrom，2025））通过整个训练过程对每个预测进行微分，并计算其相对于训练数据的精确影响，但这种方法需要对每个预测单独运行一次训练。为降低这一成本，我们将预算受限的归因问题转化为从少量测量中估计一个大型影响矩阵的问题。我们表明，最适合恢复该矩阵的测量方式与最适合归因本身的测量方式有所不同。随后，我们提出了两种算法——MAGE和SPELL，分别适用于矩阵重建和归因任务，它们可在现有元梯度机制上运行且不增加额外成本。实证研究表明，在不同训练规模和测量预算下，我们的方法相较于现有基线表现出强劲的性能。

英文摘要

Data attribution seeks to quantify how individual training examples shape a model's predictions and underpins problems including data valuation, machine unlearning, and model interpretability. Despite having a long line of work, computationally scalable methods often struggle to predict the effect of removing training data in neural networks due to their non-convex nature. To overcome this challenge, metagradient-based methods such as MAGIC (Ilyas and Engstrom, 2025) differentiate each prediction through the entire training run and compute its exact influence with respect to the training data, but require a separate run for every prediction. To reduce this cost, we cast budgeted attribution as estimating a large influence matrix from a small number of measurements. We show that the measurements most appropriate for recovering this matrix differ from those best suited for attribution itself. We then present two algorithms, MAGE and SPELL, suited for reconstruction and attribution respectively, that run on existing metagradient machinery at no extra cost. Empirical studies demonstrate strong performance over existing baselines across training scales and measurement budgets.

发表机构

• Columbia University（哥伦比亚大学）
• Carnegie Mellon University（卡内基梅隆大学）

机构由 AI 辅助整理，请以论文原文为准。