CuriousLLM:通过 LLM 增强的知识图谱推理提升多文档问答
CuriousLLM: Elevating Multi-Document Question Answering with LLM-Enhanced Knowledge Graph Reasoning
AI总结:
本文提出 CuriousLLM,通过好奇心驱动的后续问题生成机制引导知识图谱检索,并构建 Follow-upQA 数据集,以更低计算成本和延迟提升多文档问答性能。
摘要:
大语言模型(LLM)已在开放域问答中取得显著成功。然而,它们仍面临幻觉和知识截止等挑战。通过在生成答案之前为 LLM 提供相关上下文,情境学习可以缓解这些问题。近期文献提出了 Knowledge Graph Prompting(KGP,知识图谱提示),该方法将知识图谱与基于 LLM 的遍历智能体相结合,以显著提升文档检索质量。但是,KGP 需要使用大型数据集进行高成本微调,并且仍容易产生幻觉。本文提出 CuriousLLM,这是一种将好奇心驱动的推理机制集成到 LLM 智能体中的增强方法。该机制使智能体能够生成相关的后续问题,从而更高效地引导信息检索过程。我们方法的核心是构建新的 Follow-upQA 数据集,该数据集将问题和支持性证据作为输入,并将后续问题作为真实标签。这些后续问题要么询问为完整回答用户查询仍缺少什么信息,要么使用特殊标记表示检索到的证据已经充分。实验表明,CuriousLLM 显著提升了 LLM 在多文档问答(MD-QA)中的性能,同时避免了原始 KGP 框架带来的巨大计算成本和延迟。
英文摘要:
Large Language Models (LLMs) have achieved significant success in open-domain question answering. However, they continue to face challenges such as hallucinations and knowledge cutoffs. These issues can be mitigated through in-context learning by providing LLMs with relevant context before generating answers. Recent literature proposes Knowledge Graph Prompting (KGP) which integrates knowledge graphs with an LLM-based traversal agent to substantially enhance document retrieval quality. However, KGP requires costly fine-tuning with large datasets and remains prone to hallucination. In this paper, we propose CuriousLLM, an enhancement that integrates a curiosity-driven reasoning mechanism into an LLM agent. This mechanism enables the agent to generate relevant follow-up questions, thereby guiding the information retrieval process more efficiently. Central to our approach is the development of the new Follow-upQA dataset, which includes questions and supporting evidence as input, with follow-up questions serving as ground truths. These follow-up questions either inquire about what is still missing to fully answer the user's query or use special tokens to signify that the retrieved evidence is sufficient. Our experiments show that CuriousLLM significantly boosts LLM performance in multi-document question answering (MD-QA), circumventing the substantial computational costs and latency from the original KGP framework.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。