arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.03783cs.LGcs.AIcs.CL
收藏

高效估计用于任务归因的核替代模型

Efficient Estimation of Kernel Surrogate Models for Task Attribution

Northeastern University(东北大学)

Zhenshuo Zhang, Minxuan Duan, Hongyang R. Zhang

AI总结:

本文提出核替代模型用于高效估计任务归因,通过第二阶分析建立线性替代模型与影响函数的联系,并在多个任务设置中验证其有效性,实现更高的相关性和可扩展性。

摘要:

现代AI代理如大语言模型同时在翻译、代码生成、数学推理和文本预测等多样化任务上进行训练。关键问题是如何量化每个训练任务对目标任务性能的影响,即任务归因问题。直接方法leave-one-out重训练测量每个任务的效应,但计算上不可行。替代方法通过构建替代模型预测任意训练任务子集的目标任务性能已出现。先前工作集中在线性替代模型上,捕捉一阶关系但遗漏非线性交互如XOR效应。本文首先考虑统一的任务加权框架分析任务归因方法,并通过第二阶分析建立线性替代模型与影响函数的新联系。然后引入核替代模型,更有效地表示第二阶任务交互。为高效学习核替代模型,我们开发了基于梯度的估计程序,利用预训练模型的一阶近似;实验证明,这种程序在不重复重训练的情况下,能以小于2%的相对误差获得准确的替代估计。在多个设置中进行实验,包括transformers中的数学推理、上下文学习和多目标强化学习,验证了核替代模型的有效性。它们与leave-one-out地面真实值的相关性比线性替代模型和影响函数基线高25%,使任务归因更准确和可扩展。当用于下游数据选择时,核替代模型在上述设置中进一步提高40%。

英文摘要:

Modern AI agents such as large language models are trained on diverse tasks -- translation, code generation, mathematical reasoning, and text prediction -- simultaneously. A key question is how to quantify the influence of each individual training task on performance on a target task, a problem we refer to as task attribution. The direct approach, leave-one-out retraining, measures the effect of removing each task, but is computationally infeasible at scale. An alternative approach that builds surrogate models to predict the performance on a target task for any subset of training tasks has emerged in the recent literature. Prior work focuses on linear surrogate models, which capture first-order relationships but miss nonlinear interactions such as XOR-type effects. In this paper, we first consider a unified task-weighting framework for analyzing task-attribution methods and establish a new connection between linear surrogate models and influence functions via a second-order analysis. Then, we introduce kernel surrogate models, which more effectively represent second-order task interactions. To efficiently learn the kernel surrogate, we develop a gradient-based estimation procedure that leverages a first-order approximation of pretrained models; empirically, this yields accurate surrogate estimates with less than $2\%$ relative error without repeated retraining. Experiments across multiple settings -- including mathematical reasoning in transformers, in-context learning, and multi-objective reinforcement learning -- demonstrate the effectiveness of kernel surrogate models. They achieve a $25\%$ higher correlation with the leave-one-out ground truth than linear surrogates and influence-function baselines, enabling more accurate and scalable task attribution. When used for downstream data selection, kernel surrogate models further yield a $40\%$ improvement in the aforementioned settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑