arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.08877cs.CLcs.AI
收藏

虚构还是记忆?大型语言模型中概率学习的两面

Hallucinate or Memorize? The Two Sides of Probabilistic Learning in Large Language Models

Meijo University(名古屋大学)

Junichiro Niimi

AI总结:

针对LLM在引文推荐中存在虚构不存在论文的问题,本研究假设引文数量可代理训练数据冗余,通过GPT-4.1实验发现引文数量与事实准确性高度相关,超千次被引论文几乎被逐字记忆且相似内容会引发记忆干扰,揭示了LLM概率学习中生成与记忆的两面性。

摘要:

大型语言模型(LLMs)已被越来越多地应用于从自然语言理解到代码生成的广泛任务中。尽管它们也被用于辅助引文推荐,但虚构不存在的论文仍是一个主要问题。基于先前研究,本研究假设LLM正确生成书目记录的能力取决于底层知识是生成的还是记忆的,高被引论文(即更频繁出现在预训练语料库中)的虚构率更低。因此,我们假设引文数量可作为训练数据冗余(即给定书目记录在预训练语料库中出现的频率)的代理指标,并研究引文频率如何影响LLM输出中的虚构参考文献。我们使用GPT-4.1,在20个计算机科学领域生成并手动验证了100条引文,并通过生成元数据与真实元数据之间的余弦相似度来衡量事实一致性。结果表明:(i)引文数量与事实准确性高度相关;(ii)当引文数量超过约1000次时,书目信息几乎被逐字记忆;(iii)当多篇高被引论文内容相似时,会发生记忆干扰。这些发现表明存在一个从泛化转向记忆的阈值,高被引论文几乎被逐字保留在模型中。

英文摘要:

Large language models (LLMs) have been increasingly applied to a wide range of tasks, from natural language understanding to code generation. While they have also been used to assist in citation recommendation, the hallucination of non-existent papers remains a major issue. Building on prior studies, this study hypothesizes that an LLM's ability to correctly produce bibliographic records depends on whether the underlying knowledge is generated or memorized, with highly cited papers (i.e., more frequently appear in the pretraining corpus) showing lower hallucination rates. We therefore assume citation count as a proxy for training data redundancy (i.e., the frequency with which a given bibliographic record appears in the pretraining corpus) and investigate how citation frequency affects hallucinated references in LLM outputs. Using GPT-4.1, we generated and manually verified 100 citations across twenty computer-science domains, and measured factual consistency via cosine similarity between generated and authentic metadata. The results revealed that (i) citation count is strongly correlated with factual accuracy, (ii) bibliographic information becomes almost verbatim memorized beyond roughly 1,000 citations, and (iii) memory interference occurs when multiple highly cited papers share similar content. These findings indicate a threshold where generalization shifts into memorization, with highly cited papers being nearly verbatim retained in the model.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑