arXiv 2411.115312025-01-14cs.CLcs.AI

以知识图谱嵌入作为额外模态解决语言模型中的幻觉问题

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality

Institute for Clarity in Documentation（文献清晰研究所）；Inria Paris-Rocquencourt（法国国家信息与自动化研究所巴黎-罗康校区）；Rajiv Gandhi University（拉吉夫·甘地大学）；Tsinghua University（清华大学）；Palmer Research Laboratories（帕尔默研究实验室）

Viktoriia Chekalina, Anton Razzhigaev, Elizaveta Goncharova, Andrey Kuznetsov

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本研究提出将知识图谱嵌入作为额外模态，通过训练适配器整合到语言模型中，无需外部检索与模型微调，构建了含300万样本的WikiEntities数据集，在多个基准上验证了其减少大语言模型幻觉、提升事实准确性的效果。

摘要：

本文提出了一种通过引入知识图谱（KGs）作为额外模态来减少大语言模型（LLMs）幻觉的方法。该方法将输入文本转换为一组知识图谱嵌入，并使用适配器将这些嵌入整合到语言模型空间中，无需依赖外部检索流程。\n 为支撑该研究，我们构建了WikiEntities数据集，其中包含超过300万篇标注了Wikidata实体及对应PyTorch-BigGraph嵌入的维基百科文本。该数据集可作为训练实体链接模型、以及通过专用适配器将所述方法适配到各类大语言模型的宝贵资源。\n 我们的方法无需对语言模型本身进行微调，仅需训练适配器，这确保了模型在其他任务上的性能不受影响。我们使用该数据集分别为Mistral 7B、LLaMA 2-7B（chat版）和LLaMA 3-8B（instruct版）模型训练了适配器，并验证了该方法在HaluEval、True-False基准测试以及FEVER数据集上的性能提升。结果表明，将知识图谱作为新模态引入可有效减少幻觉，提升语言模型的事实准确性，且全程无需外部检索。

英文摘要：

In this paper we present an approach to reduce hallucinations in Large Language Models (LLMs) by incorporating Knowledge Graphs (KGs) as an additional modality. Our method involves transforming input text into a set of KG embeddings and using an adapter to integrate these embeddings into the language model space, without relying on external retrieval processes. To facilitate this, we created WikiEntities, a dataset containing over 3 million Wikipedia texts annotated with entities from Wikidata and their corresponding embeddings from PyTorch-BigGraph. This dataset serves as a valuable resource for training Entity Linking models and adapting the described method to various LLMs using specialized adapters. Our method does not require fine-tuning of the language models themselves; instead, we only train the adapter. This ensures that the model's performance on other tasks is not affected. We trained an adapter for the Mistral 7B, LLaMA 2-7B (chat), and LLaMA 3-8B (instruct) models using this dataset and demonstrated that our approach improves performance on the HaluEval, True-False benchmarks and FEVER dataset. The results indicate that incorporating KGs as a new modality can effectively reduce hallucinations and improve the factual accuracy of language models, all without the need for external retrieval.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。