arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2112.03052cs.LGcs.CLcs.CV
收藏

放大影响函数

Scaling Up Influence Functions

Google AI(谷歌人工智能)

Andrea Schioppa, Polina Zablotskaia, David Vilar, Artem Sokolov

摘要:

我们解决了有效的计算影响函数的跟踪预测回训练数据。提出并分析了一种基于Arnoldi迭代的Hessian逆计算加速方法。通过这种改进,据我们所知,我们实现了影响函数的首次成功实现,该影响函数可扩展到具有数亿参数的全尺寸(语言和视觉)Transformer模型。我们评估我们的方法对图像分类和序列到序列的任务与数以千万计的训练样本。我们的代码将在https://github.com/google-research/jax-influence上提供。

英文摘要:

We address efficient calculation of influence functions for tracking predictions back to the training data. We propose and analyze a new approach to speeding up the inverse Hessian calculation based on Arnoldi iteration. With this improvement, we achieve, to the best of our knowledge, the first successful implementation of influence functions that scales to full-size (language and vision) Transformer models with several hundreds of millions of parameters. We evaluate our approach on image classification and sequence-to-sequence tasks with tens to a hundred of millions of training examples. Our code will be available at https://github.com/google-research/jax-influence.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑