arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.12979cs.CL
收藏

面向知识密集型问答的内部与外部知识交互精炼框架

Internal and External Knowledge Interactive Refinement Framework for Knowledge-Intensive Question Answering

Haowei Du, Dongyan Zhao

AI总结:

本文提出IEKR框架,利用LLM内部知识辅助外部知识检索,并用外部知识精炼内部幻觉,在3个基准数据集上达到SOTA。

摘要:

近期的研究尝试将外部知识整合到LLMs中,以解决LLM生成内容的局限性和潜在事实错误。然而,如何从大量外部知识中检索到正确的知识是一个挑战。为此,我们通过实验观察到,LLMs在其预训练参数中已编码了丰富的知识,在将其应用于知识密集型任务时,利用这些内部知识能改善外部知识的检索。在本文中,我们提出了一种新的内部与外部知识交互精炼范式(IEKR),利用LLM中的内部知识帮助从外部知识库检索相关知识,并利用外部知识精炼生成内部知识时产生的幻觉。通过向LLMs添加类似“告诉我关于”的提示,我们尝试审查相关的显式知识,并将其与查询一起插入检索器以进行外部检索。外部知识被用来补充内部知识,并作为LLM的输入以生成答案。我们在知识密集型问答任务中的3个基准数据集上,使用不同的LLMs和领域进行了实验,取得了新的SOTA。进一步分析表明了我们方法中不同模块的有效性。

英文摘要:

Recent works have attempted to integrate external knowledge into LLMs to address the limitations and potential factual errors in LLM-generated content. However, how to retrieve the correct knowledge from the large amount of external knowledge imposes a challenge. To this end, we empirically observe that LLMs have already encoded rich knowledge in their pretrained parameters and utilizing these internal knowledge improves the retrieval of external knowledge when applying them to knowledge-intensive tasks. In this paper, we propose a new internal and external knowledge interactive refinement paradigm dubbed IEKR to utilize internal knowledge in LLM to help retrieve relevant knowledge from the external knowledge base, as well as exploit the external knowledge to refine the hallucination of generated internal knowledge. By simply adding a prompt like 'Tell me something about' to the LLMs, we try to review related explicit knowledge and insert them with the query into the retriever for external retrieval. The external knowledge is utilized to complement the internal knowledge into input of LLM for answers. We conduct experiments on 3 benchmark datasets in knowledge-intensive question answering task with different LLMs and domains, achieving the new state-of-the-art. Further analysis shows the effectiveness of different modules in our approach.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑