arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.26808cs.LGcs.AIcs.ITmath.IT
收藏

创新:幻觉的几乎刻画

Innovation: An Almost Characterization of Hallucination

School of Technology and Computer Science, Tata Institute of Fundamental Research, Mumbai, Maharashtra - 400 005, India(技术与计算机科学学院,塔塔基础研究机构,孟买,马哈拉施特拉邦 - 400 005, 印度)

Nishant P. Das, Piyush Srivastava

AI总结:

本文引入“创新”属性来刻画大语言模型幻觉的必然性,证明创新与幻觉几乎等价,并基于创新率给出新的幻觉率下界。

摘要:

幻觉是大语言模型(LLMs)的一个核心局限,大量工作致力于理解和缓解它。为此,Kalai 和 Vempala(STOC 2024)引入了一个概率框架来形式化校准和幻觉,并证明高概率下,校准的 LLM 大致以“缺失质量”(衡量训练数据相对于其来源的不完整程度)的速率产生幻觉。这引出了两个基本问题:(i) 校准的 LLM 的什么属性使得幻觉不可避免?(ii) 能否通过放弃校准来避免幻觉?我们通过引入一个更简单的属性——我们称之为“创新”——来回答这些问题,该属性衡量模型产生训练数据之外输出的倾向。我们证明,创新由 Kalai 和 Vempala 识别的幻觉条件蕴含,并且进一步,它是幻觉的几乎刻画:幻觉蕴含创新,反之,创新高概率地蕴含幻觉。我们还基于“创新率”给出了幻觉率的下界,并通过将创新率与缺失质量联系起来,获得了基于缺失质量的新的幻觉率下界,扩展了 Kalai 和 Vempala 的结果。

英文摘要:

Hallucination is a central limitation of large language models (LLMs), and substantial effort has been devoted to understanding and mitigating it. Towards this, Kalai and Vempala (STOC 2024) introduced a probabilistic framework formalizing calibration and hallucination, and showed that, with high probability, calibrated LLMs hallucinate roughly at the rate of the "missing mass", a measure of how incomplete the training data is relative to its source. This raises two fundamental questions: (i) what property of a calibrated LLM makes hallucinations unavoidable? and (ii) can hallucinations be avoided by giving up calibration? We answer these questions by introducing a simpler property we call innovation that measures the tendency of a model to produce outputs outside the training data. We show that innovation is implied by the condition for hallucination identified by Kalai and Vempala, and, further, that it is an almost characterization of hallucination: hallucination implies innovation, and conversely, innovation implies hallucination with high probability. We also provide lower bounds on the hallucination rate based on the "innovation rate", and by relating innovation rate back to missing mass, we obtain new hallucination rate lower bounds based on missing mass that extend the results of Kalai and Vempala.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑