arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2605.01428cs.CL

幻觉削弱信任;元认知是前进的方向

Hallucinations Undermine Trust; Metacognition is a Way Forward

Tel Aviv University(特拉维夫大学)

Gal Yona, Mor Geva, Yossi Matias

AI总结:

研究指出,生成AI的幻觉问题源于知识边界扩展而非意识提升,提出通过元认知表达不确定性以提升可信度和能力。

摘要:

尽管事实可靠性有所提高,幻觉问题仍困扰生成AI,尤其在复杂场景中。本文认为事实性提升主要来自扩展知识边界而非提升边界意识。提出通过元认知表达不确定性,即在语言上与内在不确定性一致,以实现可信且有能力的AI系统。

英文摘要:

Despite significant strides in factual reliability, errors -- often termed hallucinations -- remain a major concern for generative AI, especially as LLMs are increasingly expected to be helpful in more complex or nuanced setups. Yet even in the simplest setting -- factoid question-answering with clear ground truth-frontier models without external tools continue to hallucinate. We argue that most factuality gains in this domain have come from expanding the model's knowledge boundary (encoding more facts) rather than improving awareness of that boundary (distinguishing known from unknown). We conjecture that the latter is inherently difficult: models may lack the discriminative power to perfectly separate truths from errors, creating an unavoidable tradeoff between eliminating hallucinations and preserving utility. This tradeoff dissolves under a different framing. If we understand hallucinations as confident errors -- incorrect information delivered without appropriate qualification -- a third path emerges beyond the answer-or-abstain dichotomy: expressing uncertainty. We propose faithful uncertainty: aligning linguistic uncertainty with intrinsic uncertainty. This is one facet of metacognition -- the ability to be aware of one's own uncertainty and to act on it. For direct interaction, acting on uncertainty means communicating it honestly; for agentic systems, it becomes the control layer governing when to search and what to trust. Metacognition is thus essential for LLMs to be both trustworthy and capable; we conclude by highlighting open problems for progress towards this objective.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑