arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.06121cs.CL
收藏

少即是多:让更小的语言模型成为多跳知识图谱问答(KGQA)的合格子图检索器

Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA

School of Informatics, University of Edinburgh(爱丁堡大学信息学院);Huawei Edinburgh Research Centre(华为爱丁堡研究中心);The Chinese University of Hong Kong(香港中文大学)

Wenyu Huang, Guancheng Zhou, Hongru Wang, Pavlos Vougiouklis, Mirella Lapata, Jeff Z. Pan

AI总结:

针对多跳KGQA的子图检索需求,提出将子图检索建模为小语言模型的条件生成任务,用关系序列特殊token表示子图,小模型即可取得媲美大模型的检索效果,3B模型搭配LLM读取器还刷新了两个基准的端到端SOTA性能。

摘要:

检索增强生成(RAG)被广泛用于向大语言模型(LLM)注入外部非参数知识。近期研究表明,知识图谱(KGs)蕴含着对大语言模型有价值的外部知识。从知识图谱中检索信息与从文档集合中提取信息存在差异。现有大多数方法致力于直接检索相关子图,从而省去语义解析方法传统上所需的大量SPARQL标注工作。本文将子图检索任务建模为由小语言模型处理的条件生成任务。具体而言,我们将子图标识符定义为关系序列,每个关系都表示为存储在语言模型中的特殊token。我们的基础生成式子图检索模型仅包含2.2亿(220M)参数,与依赖70亿(7B)参数的当前最优模型相比,取得了具有竞争力的检索性能,证明小语言模型具备执行子图检索任务的能力。此外,我们最大的30亿(3B)参数模型搭配LLM读取器后,在WebQSP和CWQ两个基准上均刷新了端到端性能的最优(SOTA)纪录。我们的模型和数据将在以下网址公开:https://github.com/hwy9855/GSR。

英文摘要:

Retrieval-Augmented Generation (RAG) is widely used to inject external non-parametric knowledge into large language models (LLMs). Recent works suggest that Knowledge Graphs (KGs) contain valuable external knowledge for LLMs. Retrieving information from KGs differs from extracting it from document sets. Most existing approaches seek to directly retrieve relevant subgraphs, thereby eliminating the need for extensive SPARQL annotations, traditionally required by semantic parsing methods. In this paper, we model the subgraph retrieval task as a conditional generation task handled by small language models. Specifically, we define a subgraph identifier as a sequence of relations, each represented as a special token stored in the language models. Our base generative subgraph retrieval model, consisting of only 220M parameters, achieves competitive retrieval performance compared to state-of-the-art models relying on 7B parameters, demonstrating that small language models are capable of performing the subgraph retrieval task. Furthermore, our largest 3B model, when plugged with an LLM reader, sets new SOTA end-to-end performance on both the WebQSP and CWQ benchmarks. Our model and data will be made available online: https://github.com/hwy9855/GSR.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑