arXiv 2407.161272024-07-23cs.CLcs.AI

使用判别指令微调生成式大型语言模型以完成知识图谱补全

Finetuning Generative Large Language Models with Discrimination Instructions for Knowledge Graph Completion

State Key Laboratory for Novel Software Technology, Nanjing University（南京大学计算机软件新技术国家重点实验室）；National Institute of Healthcare Data Science, Nanjing University（南京大学国家卫生健康数据科学研究院）

Yang Liu, Xiaobin Tian, Zequn Sun, Wei Hu

备注：: Accepted in the 23rd International Semantic Web Conference (ISWC 2024)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出微调框架DIFT，通过轻量级模型获取候选实体并利用判别指令微调LLM进行选择，结合截断采样和KG嵌入注入，有效避免锚定错误并提升知识图谱补全性能。

摘要：

传统知识图谱（KG）补全模型通过学习嵌入来预测缺失事实。近期研究尝试利用大型语言模型（LLM）以文本生成方式补全KG。然而，这些方法需要将LLM的输出锚定到KG实体上，这不可避免地会带来错误。本文提出了一种微调框架DIFT，旨在释放LLM的KG补全能力并避免锚定错误。给定一个不完整的事实，DIFT采用轻量级模型获取候选实体，并利用判别指令微调LLM，以从给定候选中选出正确实体。为在减少指令数据的同时提升性能，DIFT使用截断采样方法选择有用事实进行微调，并将KG嵌入注入LLM。在基准数据集上的大量实验证明了所提框架的有效性。

英文摘要：

Traditional knowledge graph (KG) completion models learn embeddings to predict missing facts. Recent works attempt to complete KGs in a text-generation manner with large language models (LLMs). However, they need to ground the output of LLMs to KG entities, which inevitably brings errors. In this paper, we present a finetuning framework, DIFT, aiming to unleash the KG completion ability of LLMs and avoid grounding errors. Given an incomplete fact, DIFT employs a lightweight model to obtain candidate entities and finetunes an LLM with discrimination instructions to select the correct one from the given candidates. To improve performance while reducing instruction data, DIFT uses a truncated sampling method to select useful facts for finetuning and injects KG embeddings into the LLM. Extensive experiments on benchmark datasets demonstrate the effectiveness of our proposed framework.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。