面向知识图谱补全的大语言模型语境化蒸馏
Contextualization Distillation from Large Language Model for Knowledge Graph Completion
AI总结：
本文提出语境化蒸馏策略，利用大语言模型将结构化三元组转化为上下文丰富片段，并通过重建与语境化两个辅助任务提升知识图谱补全模型的性能与可解释性。
摘要：
尽管文本信息显著提升了预训练语言模型（PLMs）在知识图谱补全（KGC）中的性能，但现有从维基百科文章或同义词集定义中收集的语料具有静态性和噪声性，往往限制了基于PLM的KGC模型的潜力。为克服这些挑战，我们提出了语境化蒸馏策略，这是一种通用的即插即用方法，兼容判别式和生成式KGC框架。我们的方法首先指示大语言模型（LLMs）将紧凑的结构化三元组转换为上下文丰富的片段。随后，我们引入两个定制的辅助任务，即重建和语境化，使较小的KGC模型能够从这些丰富化的三元组中吸收见解。在多个数据集和多种KGC技术上的综合评估突显了我们方法的有效性和适应性，揭示了无论底层流程或架构如何，性能都能得到一致的提升。此外，我们的分析使我们的方法更具可解释性，并为生成路径选择以及合适的蒸馏任务选择提供了见解。本工作的所有代码和数据将在https://github.com/David-Li0406/Contextulization-Distillation发布。
英文摘要：
While textual information significantly enhances the performance of pre-trained language models (PLMs) in knowledge graph completion (KGC), the static and noisy nature of existing corpora collected from Wikipedia articles or synsets definitions often limits the potential of PLM-based KGC models. To surmount these challenges, we introduce the Contextualization Distillation strategy, a versatile plug-in-and-play approach compatible with both discriminative and generative KGC frameworks. Our method begins by instructing large language models (LLMs) to transform compact, structural triplets into context-rich segments. Subsequently, we introduce two tailored auxiliary tasks, reconstruction and contextualization, allowing smaller KGC models to assimilate insights from these enriched triplets. Comprehensive evaluations across diverse datasets and KGC techniques highlight the efficacy and adaptability of our approach, revealing consistent performance enhancements irrespective of underlying pipelines or architectures. Moreover, our analysis makes our method more explainable and provides insight into generating path selection, as well as the choosing of suitable distillation tasks. All the code and data in this work will be released at https://github.com/David-Li0406/Contextulization-Distillation
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
