arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.01312cs.LG
收藏

不完美影响,保留排名:数据归因的TRAK理论

Understanding What TRAK Gets Right and Wrong: A Theory of Data Attribution

Columbia University(哥伦比亚大学)

Han Tong, Haolin Zou, Shubhangi Ghosh, Arian Maleki

AI总结:

本文理论分析了TRAK算法,证明其在数据归因中尽管存在误差,但能保持数据点的相对排名。

摘要:

数据归因,即追溯模型预测回特定训练数据,是解释复杂人工智能模型的重要工具。广泛使用的TRAK算法通过首先用核机近似底层模型,然后利用用于近似留一法(ALO)风险的技术来解决这一挑战。尽管其在经验上表现优异,但TRAK近似在何种理论条件下是准确的,以及在何种情况下会失效,仍鲜有研究。在本文中,我们对TRAK算法进行了理论分析,刻画其性能,并量化了该方法所依赖的近似引入的误差。我们证明,尽管近似引入了显著误差,TRAK估计的影响仍与原始影响高度相关,因此在数据点的相对排名上基本保持不变。我们通过广泛的模拟和实证研究验证了我们的理论结果。

英文摘要:

Data attribution, which traces a model's prediction back to specific training data, is an important tool for interpreting sophisticated AI models. The widely used TRAK algorithm addresses this challenge by first approximating the underlying model with a kernel machine and then leveraging techniques developed for approximating the leave-one-out (ALO) risk. Despite its strong empirical performance, the theoretical conditions under which TRAK approximations are accurate, as well as the regimes in which they break down, remain largely unexplored. In this paper, we provide a theoretical analysis of the TRAK algorithm, characterizing its performance and quantifying the errors introduced by the approximations on which the method relies. We show that although the approximations can incur significant errors, TRAK preserves the separation between highly influential and weakly influential data points. We corroborate our theoretical results through extensive simulations and empirical studies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑