arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.08435cs.CLcs.AI
收藏

大语言模型中上下文主导而参数记忆跟随的现象

When Context Leads but Parametric Memory Follows in Large Language Models

Portland State University(波特兰州立大学)

Yufei Tao, Adam Hiatt, Erik Haake, Antonie J. Jetter, Ameeta Agrawal

AI总结:

本文通过新数据集WikiAtomic研究九个LLM在知识一致场景下的知识分配,发现模型稳定依赖约70%上下文和30%参数知识,且增大上下文可减少幻觉,为优化上下文利用提供依据。

摘要:

大语言模型(LLMs)在利用多样化知识来源方面已展现出显著进展。本研究考察了九个广泛使用的LLM在知识一致场景下回答开放式问题时,如何在局部上下文与全局参数之间分配知识。我们引入了一个新数据集WikiAtomic,并系统地改变上下文大小,以分析LLM在知识一致场景中如何优先处理并利用所提供的信息及其参数化知识。此外,我们还研究了它们在不同上下文大小下产生幻觉的倾向。我们的发现揭示了跨模型的一致模式,包括对上下文知识(约70%)和参数化知识(约30%)的持续依赖,以及随着上下文增大幻觉现象的减少。这些见解凸显了更有效的上下文组织以及开发更确定性地利用输入以实现稳健性能的模型的重要性。

英文摘要:

Large language models (LLMs) have demonstrated remarkable progress in leveraging diverse knowledge sources. This study investigates how nine widely used LLMs allocate knowledge between local context and global parameters when answering open-ended questions in knowledge-consistent scenarios. We introduce a novel dataset, WikiAtomic, and systematically vary context sizes to analyze how LLMs prioritize and utilize the provided information and their parametric knowledge in knowledge-consistent scenarios. Additionally, we also study their tendency to hallucinate under varying context sizes. Our findings reveal consistent patterns across models, including a consistent reliance on both contextual (around 70%) and parametric (around 30%) knowledge, and a decrease in hallucinations with increasing context. These insights highlight the importance of more effective context organization and developing models that use input more deterministically for robust performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑