HGOT：用于事实性评估中检索增强型上下文学习的层次化思维图
HGOT: Hierarchical Graph of Thoughts for Retrieval-Augmented In-Context Learning in Factuality Evaluation
AI总结：
本文提出 HGOT 层次化思维图框架，通过查询分解、引用质量加权投票和段落评分改进检索增强上下文学习，在多项事实性基准上取得显著或同等领先性能。
摘要：
随着大型语言模型（LLM）在众多应用中的广泛采用，事实性挑战和幻觉倾向已成为一个重要关切。为解决这一问题，尤其是在检索增强型上下文学习中，我们提出层次化思维图（HGOT），这是一种结构化的多层图方法，旨在增强上下文学习过程中相关段落的检索。该框架利用 LLM 涌现出的规划能力，采用分而治之策略将复杂查询分解为可管理的子查询。它改进了用于答案选择的自洽性多数投票，纳入最近提出的引用召回率和精确率指标来评估思维质量，从而将答案的可信度与思维质量内在地联系起来。该方法在多数投票中引入加权系统，依据答案对应思维的引用质量确定答案优先级。此外，我们提出一种用于评估检索段落的评分机制，考虑引用频率和质量、自洽性置信度以及检索模块排名等因素。实验表明，HGOT 是一种出色的通用方法：在 FEVER 上相比竞争模型最高提升 7%，并在 Open-SQuAD 上与 Retrieve-then-Read 等领先模型持平、在 HotPotQA 上与 DSP 等领先模型持平，证明了其在增强 LLM 事实性方面的有效性。
英文摘要：
With the widespread adoption of large language models (LLMs) in numerous applications, the challenge of factuality and the propensity for hallucinations has emerged as a significant concern. To address this issue, particularly in retrieval-augmented in-context learning, we introduce the hierarchical graph of thoughts (HGOT), a structured, multi-layered graph approach designed to enhance the retrieval of pertinent passages during in-context learning. The framework utilizes the emergent planning capabilities of LLMs, employing the divide-and-conquer strategy to break down complex queries into manageable sub-queries. It refines self-consistency majority voting for answer selection, which incorporates the recently proposed citation recall and precision metrics to assess the quality of thoughts, linking an answer's credibility intrinsically to the thought's quality. This methodology introduces a weighted system in majority voting, prioritizing answers based on the citation quality of their thoughts. Additionally, we propose a scoring mechanism for evaluating retrieved passages, considering factors such as citation frequency and quality, self-consistency confidence, and the retrieval module's ranking. Experiments indicate that HGOT excels as a versatile approach, outperforming competing models in FEVER by up to $7\%$ and matching leading models such as Retrieve-then-Read in Open-SQuAD, and DSP in HotPotQA, demonstrating its efficacy in enhancing LLMs' factuality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
