arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.00487cs.CL
收藏

通过将上下文整合进模型参数实现自更新的大语言模型

Self-Updatable Large Language Models by Integrating Context into Model Parameters

University of California San Diego(加利福尼亚大学圣迭戈分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)

Yu Wang, Xinshuang Liu, Xiusi Chen, Sean O'Brien, Junda Wu, Julian McAuley

AI总结:

针对大语言模型难以快速整合小规模经验的问题,提出SELF-PARAM框架,通过最小化原始模型与目标模型预测的KL散度将知识内化进参数,无需额外参数且性能优于现有方法。

摘要:

尽管大语言模型(LLMs)已取得显著进展,但快速、频繁地整合小规模经验(例如与周围物体的交互)仍是一项重大挑战。吸收这些经验的两个关键因素是:(1)效能:准确记忆近期事件的能力;(2)留存:回忆久远经验的能力。现有方法要么通过持续学习、模型编辑或知识蒸馏技术将经验嵌入模型参数,这类方法往往难以应对快速更新和复杂交互;要么依赖外部存储实现长期留存,从而增加了存储需求。本文提出SELF-PARAM(带参数整合的自更新大语言模型)。SELF-PARAM无需额外参数,同时能保证接近最优的效能和长期留存。我们的方法采用一种训练目标,最小化原始模型(可访问上下文信息)与目标模型(无此类访问权限)预测之间的Kullback-Leibler(KL)散度。通过生成与该知识相关的多样化问答对,并在该数据集上最小化KL散度,我们更新目标模型,将知识无缝内化到其参数中。在问答和对话推荐任务上的评估表明,即便考虑非零存储需求,SELF-PARAM的性能也显著优于现有方法。这一进展通过将知识直接嵌入模型参数,为大语言模型更高效、更可扩展的经验整合铺平了道路。

英文摘要:

Despite significant advancements in large language models (LLMs), the rapid and frequent integration of small-scale experiences, such as interactions with surrounding objects, remains a substantial challenge. Two critical factors in assimilating these experiences are (1) Efficacy: the ability to accurately remember recent events; (2) Retention: the capacity to recall long-past experiences. Current methods either embed experiences within model parameters using continual learning, model editing, or knowledge distillation techniques, which often struggle with rapid updates and complex interactions, or rely on external storage to achieve long-term retention, thereby increasing storage requirements. In this paper, we propose SELF-PARAM (Self-Updatable Large Language Models with Parameter Integration). SELF-PARAM requires no extra parameters while ensuring near-optimal efficacy and long-term retention. Our method employs a training objective that minimizes the Kullback-Leibler (KL) divergence between the predictions of an original model (with access to contextual information) and a target model (without such access). By generating diverse question-answer pairs related to the knowledge and minimizing the KL divergence across this dataset, we update the target model to internalize the knowledge seamlessly within its parameters. Evaluations on question-answering and conversational recommendation tasks demonstrate that SELF-PARAM significantly outperforms existing methods, even when accounting for non-zero storage requirements. This advancement paves the way for more efficient and scalable integration of experiences in large language models by embedding knowledge directly into model parameters.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑