发表机构
Institute of Intelligent Software; Institute of Software, CAS; University of Liverpool; Guangzhou Jiayi Software Technology Co., Ltd.（智能软件研究所; 中国科学院软件研究所; 利物浦大学; 广州嘉意软件科技有限公司）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究大语言模型输出不可靠问题，提出逻辑图不确定性（LGU）框架，该框架能明确建模答案间逻辑关系，通过聚合概率质量、计算熵等方式改进不确定性估计，在多个问答基准测试中优于现有方法。
AI 中文摘要
大语言模型（LLMs）常产生看似自信却不可靠的输出，给安全敏感应用的部署带来关键挑战。现有不确定性度量如语义熵仅捕捉语义等价层面的一致性，忽略不同答案间的逻辑关系。因此在生成的回答形式多样但逻辑兼容的场景中易高估不确定性并误判幻觉。我们提出逻辑图不确定性（LGU）框架，它明确对答案间的蕴含和不相容性建模。LGU沿蕴含链聚合概率质量，计算逻辑最大假设的熵并惩罚它们之间的相互不相容性。在多个问答基准测试中，LGU持续改进了现有方法的不确定性估计，在各数据集上比语义熵基线的AUROC最高提升7.1%，AUARC最高提升3.5%。
英文摘要
Large Language Models often produce confidently stated yet unreliable outputs, posing critical challenges for deployment in safety-sensitive applications. Existing uncertainty metrics such as semantic entropy capture agreement at the level of semantic equivalence, but largely ignore the logical relationships between distinct answers. As a result, they tend to overestimate uncertainty and falsely flag hallucinations in settings where generated responses are diverse in form yet logically compatible (e.g., differing only in granularity or specificity). We propose Logical Graph Uncertainty (LGU), a framework that explicitly models implication and incompatibility among answers. LGU aggregates probability mass along entailment chains onto the most specific hypotheses the answers support, measures the entropy of the resulting distribution, and penalizes mutual incompatibility among those hypotheses. Across multiple question-answering benchmarks and model families, LGU ranks first on average among existing uncertainty measures, with its largest gains---up to +7.1\% AUROC and +3.5\% AUARC over semantic entropy---on questions whose sampled answers are logically structured.
Comments21 pages, 3 figures, 11 tables. Under review
