arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.04086cs.CL
收藏

KDCM:通过显式推理结构减少大语言模型中的幻觉

KDCM: Reducing Hallucination in LLMs through Explicit Reasoning Structures

School of Computer Engineering, Jiangsu Ocean University(江苏海洋大学计算机工程学院);School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院)

Jinbo Hao, Kai Yang, Qingzhen Su, Yifan Li, Chao Jiang

AI总结:

KDCM通过显式推理结构减少大语言模型中的幻觉,提升预测可靠性与可解释性。

摘要:

为减轻大型语言模型(LLMs)中的幻觉现象,我们提出了一种框架,重点在于由提示引起的错误。我们的方法通过引入一个可编程模块扩展了链式知识蒸馏方法,该模块嵌入在推理提示中作为可执行代码,使模型能够在推理过程中利用外部结构化知识。基于此设计,我们开发了一种增强的基于蒸馏的推理框架,通过显式调节中间推理步骤,实现了更可靠的预测。我们使用GPT-4和LLaMA-3.3在多个公开基准上评估了所提出的方法。实验结果表明,代码引导的推理显著提高了上下文建模并减少了提示诱导的幻觉。具体而言,HIT@1、HIT@3和HIT@5分别增加了15.64%、13.38%和13.28%,在多个评估设置中得分超过95%。这些发现表明,所提出的方法有效约束了错误推理,同时提高了准确性和可解释性。

英文摘要:

To mitigate hallucinations in large language models (LLMs), we propose a framework that focuses on errors induced by prompts. Our method extends a chain-style knowledge distillation approach by incorporating a programmable module that guides knowledge graph exploration. This module is embedded as executable code within the reasoning prompt, allowing the model to leverage external structured knowledge during inference. Based on this design, we develop an enhanced distillation-based reasoning framework that explicitly regulates intermediate reasoning steps, resulting in more reliable predictions. We evaluate the proposed approach on multiple public benchmarks using GPT-4 and LLaMA-3.3. Experimental results show that code-guided reasoning significantly improves contextual modeling and reduces prompt-induced hallucinations. Specifically, HIT@1, HIT@3, and HIT@5 increase by 15.64%, 13.38%, and 13.28%, respectively, with scores exceeding 95% across several evaluation settings. These findings indicate that the proposed method effectively constrains erroneous reasoning while improving both accuracy and interpretability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑