用于长文本语言模型输出的基于图的不确定性指标
Graph-based Uncertainty Metrics for Long-form Language Model Outputs
AI总结：
本文提出 Graph Uncertainty，用二分图和图中心性度量长文本 LLM 生成中的声明级不确定性，并结合不确定性感知解码提升事实性与信息量。
摘要：
近期，大型语言模型（LLM）的进展显著提升了文本生成能力，但这些系统仍会产生幻觉，而针对长文本 LLM 生成内容的细粒度不确定性估计仍然具有挑战性。在本工作中，我们提出 Graph Uncertainty——它将 LLM 生成内容与其中的声明之间的关系表示为一个二分图，并使用一族图中心性指标来估计声明级别的不确定性。在这一视角下，现有基于自洽性概念的不确定性估计方法可被视为使用度中心性作为不确定性度量，而我们表明，更复杂的替代方案（如接近中心性）能在声明级不确定性估计上带来持续提升。此外，我们提出了不确定性感知解码技术，该技术利用图结构和不确定性估计，通过仅保留最可靠的声明来提高 LLM 生成内容的事实性。与现有方法相比，我们基于图的不确定性指标在多种长文本生成设置下使 AUPRC 平均相对提升 6.8%，并且我们的端到端系统相比现有解码技术在事实性上带来稳定的 2–4% 提升，同时显著提高了生成回复的信息量。
英文摘要：
Recent advancements in Large Language Models (LLMs) have significantly improved text generation capabilities, but these systems are still known to hallucinate, and granular uncertainty estimation for long-form LLM generations remains challenging. In this work, we propose Graph Uncertainty -- which represents the relationship between LLM generations and claims within them as a bipartite graph and estimates the claim-level uncertainty with a family of graph centrality metrics. Under this view, existing uncertainty estimation methods based on the concept of self-consistency can be viewed as using degree centrality as an uncertainty measure, and we show that more sophisticated alternatives such as closeness centrality provide consistent gains at claim-level uncertainty estimation. Moreover, we present uncertainty-aware decoding techniques that leverage both the graph structure and uncertainty estimates to improve the factuality of LLM generations by preserving only the most reliable claims. Compared to existing methods, our graph-based uncertainty metrics lead to an average of 6.8% relative gains on AUPRC across various long-form generation settings, and our end-to-end system provides consistent 2-4% gains in factuality over existing decoding techniques while significantly improving the informativeness of generated responses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
