arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2502.08666cs.CLcs.AI
收藏

幻觉、单事实与校准偏差:一项实证研究

Hallucination, Monofacts, and Miscalibration: An Empirical Investigation

University of Pennsylvania(宾夕法尼亚大学);Department of Computer and Information Science(计算机与信息科学系)

Miranda Muqing Miao, Michael Kearns

AI总结:

本文通过实验证明,选择性加权可有效降低大型语言模型的幻觉,同时保持准确性,揭示了优化目标间的内在矛盾。

摘要:

大型语言模型(LLMs)中的幻觉事实最近已被证明遵循由单事实率(与经典Good-Turing缺失质量估计器相关)减去模型校准偏差(Kalai & Vempala, 2024)确定的统计下界。我们首次对经典n-gram模型和微调的编码器-解码器Transformer中的这种三重关系进行了实证研究。通过从具有不同形状参数的帕累托分布中生成训练数据,我们系统地控制单事实率,并建立其与幻觉的正相关性。为了弥合理论与实践之间的差距,我们通过将人口校准偏差项(第2.1节)替换为经验性的bin-wise KL散度,推导出幻觉边界的实证类比,并确认其实际可行性。然后我们引入了选择性加权——一种简单而有效的技术,通过战略性地重复最少5%的训练示例,有目的地向模型注入校准偏差。这种干预使幻觉减少了高达40%,挑战了通用去重政策。我们的实验揭示了一个关键的权衡:选择性加权在注入前保持了准确性的预注入水平,同时显著减少了幻觉,而标准训练逐渐提高了准确性,但未能解决持续的高幻觉,表明在优化目标中存在固有的张力。

英文摘要:

Hallucinated facts in large language models (LLMs) have recently been shown to obey a statistical lower bound determined by the monofact rate (related to the classical Good-Turing missing mass estimator) minus model miscalibration (Kalai & Vempala, 2024). We present the first empirical investigation of this three-way relationship in classical n-gram models and fine-tuned encoder-decoder Transformers. By generating training data from Pareto distributions with varying shape parameters, we systematically control the monofact rates and establish its positive relationship with hallucination. To bridge theory and practice, we derive an empirical analog of the hallucination bound by replacing the population miscalibration term (Section 2.1) with an empirical bin-wise KL divergence and confirm its practical viability. We then introduce selective upweighting -- a simple yet effective technique that strategically repeats as little as 5% of training examples -- to deliberately inject miscalibration into the model. This intervention reduces hallucination by up to 40%, challenging universal deduplication policies. Our experiments reveal a critical trade-off: selective upweighting maintains pre-injection levels of accuracy while substantially reducing hallucination, whereas standard training gradually improves accuracy but fails to address persistently high hallucination, indicating an inherent tension in optimization objectives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑