发表机构
Key Laboratory of Knowledge Engineering with Big Data (Hefei University of Technology); School of Computer Science and Information Engineering, Hefei University of Technology; Nanjing University of Science and Technology（大数据知识工程重点实验室（合肥工业大学）; 合肥工业大学计算机与信息学院; 南京理工大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对大语言模型在知识图谱推理中存在的推理证据感知漂移问题，提出结构内化规则语言模型SIRLM，通过结构规则生成协调知识与参数化知识，在36个数据集上优于17种现有方法
AI 中文摘要
知识图谱推理（KGR）旨在利用知识图谱（KG）中可用的结构证据发现潜在事实，这对KGR模型的结构语义理解能力提出了挑战。近期研究表明，大语言模型（LLM）可通过灵活的上下文学习在KGR任务上取得显著进展，但KG结构上下文与LLM参数化知识之间固有的表示不一致性仍未得到充分解决。这一局限性阻碍了LLM有效感知与KG约束对齐的推理证据，从而损害了推理的有效性和忠实性，我们将此问题称为LLM在KG上的推理证据感知漂移。为解决该问题，我们提出结构内化规则语言模型（SIRLM），其以结构规则生成为核心，将结构知识的参数化学习与推理逻辑的忠实性评估相结合，使LLM能够紧密锚定KG基础的证据。具体而言，我们首先设计结构内化规则生成器（SIRG），其包含经结构关系记忆增强的上下文学习模块，以协调结构知识与参数化知识；此外，我们为SIRG配备基于结构不变性学习的KG分词器，以及基于规则约束消息传播的神经符号推理器，这些组件分别为SIRG提供可学习的结构表示和忠实的规则执行反馈。我们的SIRLM可无缝集成到标准LLM训练范式（如SFT和GRPO）中，在36个数据集上与17种最先进的KGR方法进行的大量实验表明，SIRLM具有显著的优越性。
英文摘要
Knowledge Graph Reasoning (KGR) aims to discover latent facts by leveraging the structural evidence available in KGs, posing a challenge to the structural semantic understanding capability of KGR models. Recent studies have demonstrated that Large Language Models (LLMs) can achieve remarkable progress on KGR tasks via flexible in-context learning. However, the inherent representation inconsistency between KG structural context and LLM parametric knowledge remains inadequately addressed. This limitation prevents LLMs from effectively perceiving reasoning evidence that aligns with KG constraints, which undermines both the effectiveness and faithfulness of reasoning. We refer to this problem as reasoning evidence perception drift of LLMs over KGs. To address this problem, we propose a Structure-Internalized Rule Language Model (SIRLM), which centers on structural rule generation to couple the parametric learning of structural knowledge with the faithfulness evaluation of reasoning logic, enabling LLMs to anchor tightly to KG-grounded evidence. Specifically, we first design a Structure-Internalized Rule Generator (SIRG), which incorporates an in-context learning block augmented with a structural relation memory to coordinate structural and parametric knowledge. Furthermore, we equip SIRG with a KG tokenizer based on structural invariance learning and a neuro-symbolic reasoner based on rule-constrained message propagation. These components provide SIRG with learnable structural representations and faithful rule-execution feedback, respectively. Our SIRLM can be seamlessly integrated into standard LLM training paradigms, such as SFT and GRPO. Extensive experiments against 17 state-of-the-art KGR methods on 36 datasets demonstrate the significant superiority of SIRLM.
