arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.14741cs.CLcs.AI
收藏

Generate-on-Graph:在不完整知识图谱问答中同时将LLM视作智能体与知识图谱

Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answering

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);The Hong Kong University of Science and Technology(香港科技大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院)

Yao Xu, Shizhu He, Jiabei Chen, Zihao Wang, Yangqiu Song, Hanghang Tong, Guang Liu, Kang Liu, Jun Zhao

AI总结:

针对传统KGQA评估脱离KG不完整的真实场景、LLM未有效融合内外知识的问题,提出无需训练的GoG方法,通过思考-搜索-生成框架让LLM同时承担智能体与KG角色,在IKGQA任务上性能超越现有方法。

摘要:

为解决大语言模型(LLM)存在的知识不足与幻觉问题,大量研究探索了LLM与知识图谱(KG)的融合方式。但这类方法通常在配备完整知识图谱的传统知识图谱问答(KGQA)任务上评估,这类场景中每个问题所需的全部事实三元组都被给定KG完全覆盖,LLM主要充当在KG内查找答案实体的智能体,并未有效整合自身内部知识与KG等外部知识源。实际上,KG往往存在不完整性,无法覆盖回答问题所需的全部知识。为模拟这类真实场景并评估LLM整合内外部知识的能力,我们提出利用LLM开展不完整知识图谱问答(IKGQA)任务——该场景下给定KG缺少每个问题对应的部分事实三元组——并构建了对应数据集。为处理IKGQA任务,我们提出一种无需训练的方法Generate-on-Graph(GoG),该方法可在探索KG的同时生成新的事实三元组。具体而言,GoG通过“思考-搜索-生成”框架执行推理,在IKGQA中同时将LLM视作智能体与知识图谱。在两个数据集上的实验结果表明,我们提出的GoG性能优于所有现有方法。

英文摘要:

To address the issues of insufficient knowledge and hallucination in Large Language Models (LLMs), numerous studies have explored integrating LLMs with Knowledge Graphs (KGs). However, these methods are typically evaluated on conventional Knowledge Graph Question Answering (KGQA) with complete KGs, where all factual triples required for each question are entirely covered by the given KG. In such cases, LLMs primarily act as an agent to find answer entities within the KG, rather than effectively integrating the internal knowledge of LLMs and external knowledge sources such as KGs. In fact, KGs are often incomplete to cover all the knowledge required to answer questions. To simulate these real-world scenarios and evaluate the ability of LLMs to integrate internal and external knowledge, we propose leveraging LLMs for QA under Incomplete Knowledge Graph (IKGQA), where the provided KG lacks some of the factual triples for each question, and construct corresponding datasets. To handle IKGQA, we propose a training-free method called Generate-on-Graph (GoG), which can generate new factual triples while exploring KGs. Specifically, GoG performs reasoning through a Thinking-Searching-Generating framework, which treats LLM as both Agent and KG in IKGQA. Experimental results on two datasets demonstrate that our GoG outperforms all previous methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑