通过方向性蕴含图和声明级响应增强进行大语言模型不确定性量化
LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentation
AI总结：
针对LLM响应可信度问题，提出基于蕴含概率有向图与随机游走拉普拉斯特征值的不确定性量化方法，并结合声明级响应增强，实验验证其优越性。
摘要：
大语言模型（LLM）在各领域复杂任务中展现出卓越能力，从基础问答（QA）起步，如今被用作陌生内容的决策助手或解释器。然而，由于特定领域语料的数据稀疏性或模型的幻觉问题，它们并不总是正确。鉴于此，我们应当在多大程度上信任LLM的响应？本文提出了一种评估不确定性的新方法，通过由蕴含概率构建有向图来捕捉方向性不稳定性；鉴于所构建有向图的非对称性质，我们创新性地执行Random Walk Laplacian（随机游走拉普拉斯），随后利用拉普拉斯过程导出的特征值聚合不确定性。我们还提供了一种将现有工作的语义不确定性与所提出层相结合的方法。此外，本文识别了原始响应集中的模糊性问题，并提出一种增强方法以缓解该问题；我们开展了大量实证实验，证明了所提解决方案的优越性。
英文摘要：
The Large language models (LLMs) have showcased superior capabilities in sophisticated tasks across various domains, stemming from basic question-answer (QA), they are nowadays used as decision assistants or explainers for unfamiliar content. However, they are not always correct due to the data sparsity in specific domain corpus, or the model's hallucination problems. Given this, how much should we trust the responses from LLMs? This paper presents a novel way to evaluate the uncertainty that captures the directional instability, by constructing a directional graph from entailment probabilities, and we innovatively conduct Random Walk Laplacian given the asymmetric property of a constructed directed graph, then the uncertainty is aggregated by the derived eigenvalues from the Laplacian process. We also provide a way to incorporate the existing work's semantics uncertainty with our proposed layer. Besides, this paper identifies the vagueness issues in the raw response set and proposes an augmentation approach to mitigate such a problem, we conducted extensive empirical experiments and demonstrated the superiority of our proposed solutions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
