arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.03155cs.CLcs.AI
收藏

图上辩论:一种面向大语言模型的灵活且可靠的推理框架

Debate on Graph: a Flexible and Reliable Reasoning Framework for Large Language Models

Xi’an Jiaotong University(西安交通大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Northwestern Polytechnical University(西北工业大学);Zhejiang Createlink Technology(浙江创联科技);Shandong University(山东大学)

Jie Ma, Zhitao Gao, Qi Chai, Wangchun Sun, Pinghui Wang, Hongbin Pei, Jing Tao, Lingyun Song, Jun Liu, Chen Zhang, Lizhen Cui

AI总结:

本文提出图上辩论框架DoG,通过子图聚焦机制和多角色辩论团队缓解大语言模型在知识图谱问答中长推理路径与假阳性关系的干扰,在多个数据集上显著超越现有方法。

摘要:

大语言模型(LLMs)在现实应用中可能由于缺乏相关知识而产生幻觉。相比之下,知识图谱包含广泛的多关系结构,存储着大量的符号事实。因此,将大语言模型与知识图谱集成已被广泛探索,其中知识图谱问答(KGQA)是这一集成的关键试金石。该任务要求大语言模型通过从知识图谱中检索相关三元组来回答自然语言问题。然而,现有方法面临两个重大挑战:过长的推理路径分散了答案生成的注意力,以及假阳性关系阻碍了路径细化。在本文中,我们提出了一种迭代交互式KGQA框架,利用大语言模型的交互学习能力在图谱上进行推理与辩论(DoG)。具体而言,DoG采用子图聚焦机制,允许大语言模型在每个推理步骤后进行答案尝试,从而减轻冗长推理路径的影响。另一方面,DoG利用多角色辩论团队逐步简化复杂问题,降低假阳性关系的影响。这种辩论机制确保了推理过程的可靠性。在五个公开数据集上的实验结果表明了我们架构的有效性和优越性。值得注意的是,DoG在WebQuestions和GrailQA上的准确率分别比最先进的方法ToG高出23.7%和9.1%。此外,在上述数据集上与多种大语言模型的集成实验突显了DoG的灵活性。代码可在https://github.com/reml-group/DoG获取。

英文摘要:

Large Language Models (LLMs) may suffer from hallucinations in real-world applications due to the lack of relevant knowledge. In contrast, knowledge graphs encompass extensive, multi-relational structures that store a vast array of symbolic facts. Consequently, integrating LLMs with knowledge graphs has been extensively explored, with Knowledge Graph Question Answering (KGQA) serving as a critical touchstone for the integration. This task requires LLMs to answer natural language questions by retrieving relevant triples from knowledge graphs. However, existing methods face two significant challenges: \textit{excessively long reasoning paths distracting from the answer generation}, and \textit{false-positive relations hindering the path refinement}. In this paper, we propose an iterative interactive KGQA framework that leverages the interactive learning capabilities of LLMs to perform reasoning and Debating over Graphs (DoG). Specifically, DoG employs a subgraph-focusing mechanism, allowing LLMs to perform answer trying after each reasoning step, thereby mitigating the impact of lengthy reasoning paths. On the other hand, DoG utilizes a multi-role debate team to gradually simplify complex questions, reducing the influence of false-positive relations. This debate mechanism ensures the reliability of the reasoning process. Experimental results on five public datasets demonstrate the effectiveness and superiority of our architecture. Notably, DoG outperforms the state-of-the-art method ToG by 23.7\% and 9.1\% in accuracy on WebQuestions and GrailQA, respectively. Furthermore, the integration experiments with various LLMs on the mentioned datasets highlight the flexibility of DoG. Code is available at \url{https://github.com/reml-group/DoG}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑