arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2601.13600cs.AI

基于噪声LLM预言机的全局一致性基础

Global Consistency Repair for Natural-Language Claim Sets

Nanyang Technological University(南洋理工大学);Amazon Web Services(亚马逊网络服务)

Paul He, Elke Kirschbaum, Shiva Kasiviswanathan

AI总结:

本文提出一种基于LLM的自适应分治算法,用于高效检测和定位自然语言事实集合的全局不一致问题,具有低次多项式查询复杂度。

摘要:

确保自然语言事实集合的全局一致性对于事实核查、摘要和知识库构建等任务至关重要。尽管大语言模型(LLMs)可以评估小事实子集的一致性,但其判断具有噪声性,成对检查不足以保证全局一致性。我们正式化了这一问题,证明在最坏情况下验证全局一致性需要指数级的预言机查询。为使任务实用,我们提出了一种自适应的分治算法,该算法能够识别事实的最小不一致子集(MUSes)并可选地通过击中集计算最小修复。我们的方法具有低次多项式查询复杂度。在合成和真实LLM预言机上的实验表明,我们的方法高效地检测并定位不一致,提供了一个可扩展的基于LLM评估器的语言一致性验证框架。

英文摘要:

NLP systems increasingly aggregate claims from retrieved passages, documents, and generated reasoning steps. Yet current factuality methods mostly verify claims individually or compare pairs of claims, leaving open the question of whether the entire aggregated claim set can be retained coherently. This misses a fundamental failure mode: claims may be locally plausible and pairwise compatible while still being jointly inconsistent. We formalize global consistency repair: given an inconsistent natural-language claim set, preserve the largest coherent subset while localizing the compact sources of inconsistency. We introduce QXR, an adaptive diagnosis-and-repair framework that treats an LLM as a noisy set-level consistency judge, localizes minimal conflicting subsets, and repairs the claim set by deleting a small hitting set of claims. Across LLM judges and benchmarks derived from VitaminC, FEVER, and conflicting-evidence RAG, QXR preserves substantially more valid claims than direct all-at-once judging while maintaining high precision. These results suggest that factuality in claim aggregation systems should be treated as a set-level repair problem, not only as local verification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑