arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.12656cs.CLcs.AI
收藏

用于极限多标签知识图谱链接预测的检索增强语言模型

Retrieval-Augmented Language Model for Extreme Multi-Label Knowledge Graph Link Prediction

Yu-Hsiang Lin, Huang-Ting Shieh, Chih-Yu Liu, Kuang-Ting Lee, Hsiao-Cheng Chang, Jing-Lun Yang, Yu-Sheng Lin

AI总结:

提出极限多标签知识图谱链接预测任务,并用融合实体、关系与文本的检索增强小参数语言模型,验证差异化KG增强策略和文本信息的显著性能收益。

摘要:

大型语言模型(LLM)在开放式探究中的外推面临两个关键问题:(1)幻觉;(2)昂贵的训练成本。这些问题给LLM在专业领域和个性化数据中的应用带来挑战,因为这些场景要求真实回答且微调成本较低。现有工作尝试利用来自知识图谱(KG)的信息增强较小语言模型的输入,以解决该问题。然而,它们存在两个局限:(1)无法从KG中庞大的一跳邻域提取相关信息;(2)对具有不同特征的KG采用相同的增强策略,这可能导致性能低下。此外,开放式探究通常产生多个回答,进一步使外推复杂化。我们提出一项新任务,即极限多标签KG链接预测任务,使模型能够利用结构化的真实世界知识执行具有多个响应的外推。我们的检索器通过综合考虑实体、关系和文本数据来识别相关的一跳邻居。实验表明:(1)具有不同特征的KG需要不同的增强策略;(2)利用文本数据增强语言模型的输入可显著提升任务性能。通过将检索增强框架与KG结合,我们的框架以较小的参数规模能够基于给定KG进行外推。代码可在GitHub获取:https://github.com/exiled1143/Retrieval-Augmented-Language-Model-for-Multi-Label-Knowledge-Graph-Link-Prediction.git

英文摘要:

Extrapolation in Large language models (LLMs) for open-ended inquiry encounters two pivotal issues: (1) hallucination and (2) expensive training costs. These issues present challenges for LLMs in specialized domains and personalized data, requiring truthful responses and low fine-tuning costs. Existing works attempt to tackle the problem by augmenting the input of a smaller language model with information from a knowledge graph (KG). However, they have two limitations: (1) failing to extract relevant information from a large one-hop neighborhood in KG and (2) applying the same augmentation strategy for KGs with different characteristics that may result in low performance. Moreover, open-ended inquiry typically yields multiple responses, further complicating extrapolation. We propose a new task, the extreme multi-label KG link prediction task, to enable a model to perform extrapolation with multiple responses using structured real-world knowledge. Our retriever identifies relevant one-hop neighbors by considering entity, relation, and textual data together. Our experiments demonstrate that (1) KGs with different characteristics require different augmenting strategies, and (2) augmenting the language model's input with textual data improves task performance significantly. By incorporating the retrieval-augmented framework with KG, our framework, with a small parameter size, is able to extrapolate based on a given KG. The code can be obtained on GitHub: https://github.com/exiled1143/Retrieval-Augmented-Language-Model-for-Multi-Label-Knowledge-Graph-Link-Prediction.git

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑