arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14428cs.CL
收藏

基于大语言模型的知识蒸馏的时序知识图谱推理

LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning

School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院)

Wang Xing, Wei Song, Siyu Lin, Chen Wu, Man Wang

AI总结:

本文提出一种基于大语言模型的知识蒸馏框架,用于提升时序知识图谱推理性能,通过引入大语言模型作为辅助指导者,实现更高效的模型压缩与训练。

摘要:

时序知识图谱(TKG)支持对时间变化的事实进行推理,但最先进的模型往往计算成本高且部署困难。现有的压缩和蒸馏技术大多针对静态图谱设计;直接应用于时序场景可能会忽略时间依赖的交互,导致性能下降。我们提出了一种专门针对时序知识图谱推理的LLM辅助蒸馏框架。除了传统的高容量时序教师模型外,我们还引入了一个大语言模型作为辅助指导者,提供更丰富的监督。LLM提供了广泛的背景知识和时间感知信号,使轻量级的学生模型能够更好地建模事件动态,而无需增加推理时间的复杂性。通过联合优化监督和蒸馏目标,使用分阶段对齐策略逐步整合来自两个教师的指导。在多个公开的TKG基准测试中,使用多种基础架构的广泛实验表明,所提出的方法在强蒸馏基线之上一致提升了链接预测性能,同时保持了紧凑且高效的学生成模型。结果突显了大语言模型作为有效教师的潜力,能够将时间推理能力转移到资源高效的TKG系统中。

英文摘要:

Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑