arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.20079cs.CLcs.AI
收藏

分子事实:LLM事实验证中去语境化的必要条件

Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification

The University of Texas at Austin(德克萨斯大学奥斯汀分校)

Anisha Gunjal, Greg Durrett

AI总结:

针对LLM事实核查中原子事实缺语境、长文本难核查的问题,本文提出分子事实的去语境性与最小性标准,并给出自动生成基线,证明其可在模糊场景兼顾最小性与验证准确率。

摘要:

对大语言模型(LLM)生成内容进行自动真实性验证,正被越来越广泛地用于应对幻觉。文献中的一个主要矛盾在于事实核查的粒度:较大的文本块难以核查,而命题等更原子化的事实可能缺少正确解释所需的语境。本文评估了语境在这些原子事实中的作用。我们认为完全原子化的事实并非合适的表示,并为分子事实定义了两项标准:去语境性,即其独立成立的程度;以及最小性,即为实现去语境化而添加的额外信息有多小。我们量化了去语境化对最小性的影响,随后提出一种自动生成分子事实的基线方法,目标是添加适量信息。我们将其与多种去语境化方法进行比较,发现分子事实能够在模糊场景中平衡最小性与事实验证准确率。

英文摘要:

Automatic factuality verification of large language model (LLM) generations is becoming more and more widely used to combat hallucinations. A major point of tension in the literature is the granularity of this fact-checking: larger chunks of text are hard to fact-check, but more atomic facts like propositions may lack context to interpret correctly. In this work, we assess the role of context in these atomic facts. We argue that fully atomic facts are not the right representation, and define two criteria for molecular facts: decontextuality, or how well they can stand alone, and minimality, or how little extra information is added to achieve decontexuality. We quantify the impact of decontextualization on minimality, then present a baseline methodology for generating molecular facts automatically, aiming to add the right amount of information. We compare against various methods of decontextualization and find that molecular facts balance minimality with fact verification accuracy in ambiguous settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑