arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.01518cs.CLcs.LG
收藏

重新审视动态评估:面向大型语言模型的在线适应

Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models

Google DeepMind(谷歌DeepMind)

Amal Rannen-Triki, Jorg Bornschein, Razvan Pascanu, Marcus Hutter, Andras György, Alexandre Galashov, Yee Whye Teh, Michalis K. Titsias

AI总结:

本文研究大型语言模型在测试时的在线微调(动态评估),从权重记忆与上下文扩展视角分析其样本效率、分布漂移敏感性和计算开销,并指出其模糊了上下文学习与微调的界限。

摘要:

我们考虑在测试时对语言模型参数进行在线微调的问题,这也被称为动态评估。虽然众所周知,这种方法能够提升整体预测性能,尤其是在考虑训练数据与评估数据之间存在分布偏移时,但我们在此强调一个视角:在线适应将参数转化为随时间变化的状态,并提供一种以权重中的记忆实现的上下文长度扩展形式,这更符合神经科学中的记忆概念。我们特别关注适应速度(以样本效率衡量)、对整体分布漂移的敏感性,以及执行梯度计算和参数更新所带来的计算开销。我们的实证研究就在线适应何时尤其具有价值提供了洞见。我们强调,借助在线适应,上下文学习与微调之间的概念区分变得模糊:二者都是让模型以先前观察到的 token 为条件的方法。

英文摘要:

We consider the problem of online fine tuning the parameters of a language model at test time, also known as dynamic evaluation. While it is generally known that this approach improves the overall predictive performance, especially when considering distributional shift between training and evaluation data, we here emphasize the perspective that online adaptation turns parameters into temporally changing states and provides a form of context-length extension with memory in weights, more in line with the concept of memory in neuroscience. We pay particular attention to the speed of adaptation (in terms of sample efficiency),sensitivity to the overall distributional drift, and the computational overhead for performing gradient computations and parameter updates. Our empirical study provides insights on when online adaptation is particularly interesting. We highlight that with online adaptation the conceptual distinction between in-context learning and fine tuning blurs: both are methods to condition the model on previously observed tokens.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑