arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.19010cs.CLcs.AIcs.IR
收藏

评估大型语言模型的外部与参数化知识融合

Evaluating the External and Parametric Knowledge Fusion of Large Language Models

Hao Zhang, Yuyang Zhang, Xiaoguang Li, Wenxuan Shi, Haonan Xu, Huanshuo Liu, Yasheng Wang, Lifeng Shang, Qun Liu, Yong Liu, Ruiming Tang

AI总结:

本研究针对LLMs外部与参数化知识融合问题,将融合解构为四种场景并构建系统化流水线进行实验,发现增强参数化知识能显著提升整合能力,但在记忆提取及边界确定上仍存挑战。

摘要:

将外部知识整合到大型语言模型(LLMs)中,是克服其过时且静态的参数化记忆所带来限制的一种有前景的解决方案。然而,以往的研究往往过度依赖外部知识,低估了LLMs内在参数化知识的宝贵贡献。LLMs在融合外部与参数化知识方面的效能仍有待深入探索,尤其是在外部知识不完整并需要其参数化知识补充的情况下。我们提出将知识融合解构为四种不同场景,首次对LLM在每种场景下的行为进行了全面调查。我们开发了一套系统化的数据构建与知识注入流水线来模拟这些融合场景,从而开展一系列受控实验。我们的调查表明,增强LLMs内部的参数化知识能显著提升其知识整合能力。尽管如此,我们发现在记忆与提取参数化知识,以及确定参数化知识边界方面仍存在持续挑战。我们的发现旨在为未来探索LLMs内外部与参数化知识的协调提供指导。

英文摘要:

Integrating external knowledge into large language models (LLMs) presents a promising solution to overcome the limitations imposed by their antiquated and static parametric memory. Prior studies, however, have tended to over-reliance on external knowledge, underestimating the valuable contributions of an LLMs' intrinsic parametric knowledge. The efficacy of LLMs in blending external and parametric knowledge remains largely unexplored, especially in cases where external knowledge is incomplete and necessitates supplementation by their parametric knowledge. We propose to deconstruct knowledge fusion into four distinct scenarios, offering the first thorough investigation of LLM behavior across each. We develop a systematic pipeline for data construction and knowledge infusion to simulate these fusion scenarios, facilitating a series of controlled experiments. Our investigation reveals that enhancing parametric knowledge within LLMs can significantly bolster their capability for knowledge integration. Nonetheless, we identify persistent challenges in memorizing and eliciting parametric knowledge, and determining parametric knowledge boundaries. Our findings aim to steer future explorations on harmonizing external and parametric knowledge within LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑