LLM-Guided Knowledge Distillation for Temporal Knowledge Graph Reasoning
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院)
Temporal knowledge graphs (TKGs) support reasoning over time-evolving facts, yet state-of-the-art models are often computationally heavy and costly to deploy. Existing compression and distillation techniques are largely designed for static graphs; directly applying them to temporal settings may overlook time-dependent interactions and lead to performance degradation. We propose an LLM-assisted distillation framework specifically designed for temporal knowledge graph reasoning. Beyond a conventional high-capacity temporal teacher, we incorporate a large language model as an auxiliary instructor to provide enriched supervision. The LLM supplies broad background knowledge and temporally informed signals, enabling a lightweight student to better model event dynamics without increasing inference-time complexity. Training is conducted by jointly optimizing supervised and distillation objectives, using a staged alignment strategy to progressively integrate guidance from both teachers. Extensive experiments on multiple public TKG benchmarks with diverse backbone architectures demonstrate that the proposed approach consistently improves link prediction performance over strong distillation baselines, while maintaining a compact and efficient student model. The results highlight the potential of large language models as effective teachers for transferring temporal reasoning capability to resource-efficient TKG systems.
Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
Learning from Repaired Reasoning: Root-Cause-Guided On-Policy Distillation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Software and Microelectronics, Peking University(北京大学软件与微电子学院);School of Information Technology and Management, University of International Business and Economics(对外经济贸易大学信息与技术管理学院)
Where Did This Sentence Come From? Tracing Provenance in LLM Reasoning Distillation
College of Computer Science and Technology(计算机科学与技术学院);Alibaba Cloud Computing(阿里云计算);School of Artificial Intelligence(人工智能学院);Department of Mathematics(数学系)
Knowledge Distillation for Temporal Knowledge Graph Reasoning with Large Language Models
School of Computer Science and Technology, Xidian University(西安电子科技大学计算机科学与技术学院);School of Computing and Artificial Intelligence, Southwest Jiaotong University(西南交通大学计算机与人工智能学院);School of Information Science and Engineering, Chongqing Jiaotong University(重庆交通大学信息科学与工程学院);School of Information Engineering, Chang’an University(长安大学信息工程学院)
Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学计算机科学与技术系);Alibaba Group, China(阿里巴巴集团)
Scale or Reason? A Compute-Equivalent Analysis of Reasoning Distillation
Diabolocom;Artefact Research Center;Equall;ISIA Lab, University of Mons(ISIA实验室,蒙斯大学);MICS, CentraleSupélec, Université Paris-Saclay(MICS,CentraleSupélec,巴黎萨克雷大学)
Department of Statistics and Data Science, Southern University of Science and Technology(统计与数据科学系,南方科技大学);Department of Computer Sciences, University of Wisconsin–Madison(计算机科学系,威斯康星大学麦迪逊分校)
Hán Dān Xué Bù (Mimicry) or Qīng Chū Yú Lán (Mastery)? A Cognitive Perspective on Reasoning Distillation in Large Language Models
School of Philosophy, Anhui University(安徽大学哲学学院);Institute of Neuroscience, Chinese Academy of Sciences(中国科学院神经科学研究所);Faculty of Education, University of Cambridge(剑桥大学教育学院);School of Foreign Studies, South China Normal University(华南师范大学外语学院);Division of Psychology and Language Sciences, University College London(伦敦大学学院心理学与语言科学系)