arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2511.07910cs.CL
收藏

最后一层逻辑到逻辑:赋予LLM逻辑一致的结构化知识推理能力

Last Layer Logits to Logic: Empowering LLMs with Logic-Consistent Structured Knowledge Reasoning

Zhejiang University(浙江大学);Ant Group(蚂蚁集团);ZJU-Ant Group Joint Lab of Knowledge Graph(浙江大学-蚂蚁集团知识图谱联合实验室)

Songze Li, Zhiqiang Liu, Zhaoyan Gong, Xiaoke Guo, Zhongpu Bo, Zhengke Gui, Lei Liang, Huajun Chen, Wen Zhang

AI总结:

针对LLM在结构化知识推理中的逻辑漂移问题,提出Logits-to-Logic框架,通过logits增强和过滤模块修正输出逻辑缺陷,在多个KGQA基准上取得最佳性能。

摘要:

大型语言模型(LLM)通过在大量非结构化文本上进行预训练,在自然语言推理任务中表现出色,使其能够理解自然语言中的逻辑并生成逻辑一致的响应。然而,非结构化知识与结构化知识之间的表示差异使得LLM天生难以维持逻辑一致性,导致在知识图谱问答(KGQA)等结构化知识推理任务中出现 extit{逻辑漂移}挑战。现有方法通过设计嵌入提示中的复杂工作流来指导LLM推理,以解决这一局限性。然而,这些方法仅提供输入级指导,未能从根本上解决LLM输出中的 extit{逻辑漂移}问题。此外,它们僵化的推理工作流无法适应不同的任务和知识图谱。为了增强LLM在结构化知识推理中的逻辑一致性,我们专门针对自回归生成过程中的logits输出。我们提出了 extit{Logits-to-Logic}框架,该框架将logits增强和logits过滤作为核心模块,以纠正LLM输出中的逻辑缺陷。大量实验表明,我们的方法显著提高了LLM在结构化知识推理中的逻辑一致性,并在多个KGQA基准上取得了最先进的性能。

英文摘要:

Large Language Models (LLMs) achieve excellent performance in natural language reasoning tasks through pre-training on vast unstructured text, enabling them to understand the logic in natural language and generate logic-consistent responses. However, the representational differences between unstructured and structured knowledge make LLMs inherently struggle to maintain logic consistency, leading to \textit{Logic Drift} challenges in structured knowledge reasoning tasks such as Knowledge Graph Question Answering (KGQA). Existing methods address this limitation by designing complex workflows embedded in prompts to guide LLM reasoning. Nevertheless, these approaches only provide input-level guidance and fail to fundamentally address the \textit{Logic Drift} in LLM outputs. Additionally, their inflexible reasoning workflows cannot adapt to different tasks and knowledge graphs. To enhance LLMs' logic consistency in structured knowledge reasoning, we specifically target the logits output from the autoregressive generation process. We propose the \textit{Logits-to-Logic} framework, which incorporates logits strengthening and logits filtering as core modules to correct logical defects in LLM outputs. Extensive experiments show that our approach significantly improves LLMs' logic consistency in structured knowledge reasoning and achieves state-of-the-art performance on multiple KGQA benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑