arXiv 2506.029732026-01-11cs.CL

在推断前扩展：通过提前层插值增强大语言模型的事实性

Expanding before Inferring: Enhancing Factuality in Large Language Models through Premature Layers Interpolation

Sun Yat-Sen University（中山大学）；Shenzhen MSU-BIT University（深圳MSU-BIT大学）；The Hong Kong Polytechnic University（香港理工大学）；UNSW, Sydney, NSW 2052, Australia（新南威尔士大学）；Shenzhen Key Laboratory for High Performance Data Mining, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（深圳高性能数据挖掘重点实验室，深圳先进技术研究院，中国科学院）

Dingwei Chen, Ziqiang Liu, Feiteng Fang, Chak Tou Leong, Shiwen Ni, Ahmadreza Argha, Hamid Alinejad-Rokny, Min Yang, Chengming Li

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出PLI方法，通过提前层插值提升大语言模型的事实一致性，有效减少幻觉并优于现有方法。

摘要：

大语言模型（LLMs）在文本理解和生成方面表现出卓越的能力。然而，它们产生事实性不一致输出的倾向，通常被称为'幻觉'，仍然是一个关键挑战。现有方法，如基于检索和推理时校正的方法，主要在输入或输出层面解决这个问题，往往忽略了内在信息细化过程和提前层的作用。同时，对齐和微调方法资源消耗大。在本文中，我们提出了PLI（提前层插值），一种新颖的、免训练且即插即用的干预方法，旨在增强事实性。PLI通过数学插值与相邻层形成提前层来缓解幻觉。受稳定扩散和采样步骤的启发，PLI扩展了LLM的信息处理和传输深度，提高了事实一致性。在四个公开可用数据集上的实验表明，PLI有效减少了幻觉，同时在大多数情况下优于现有基线。进一步分析表明，层插值的成功与LLM内部机制密切相关。我们的数据集和代码可在https://github.com/CuSO4-Chen/PLI上获得。

英文摘要：

Large Language Models (LLMs) demonstrate remarkable capabilities in text understanding and generation. However, their tendency to produce factually inconsistent outputs, commonly referred to as ''hallucinations'', remains a critical challenge. Existing approaches, such as retrieval-based and inference-time correction methods, primarily address this issue at the input or output level, often overlooking the intrinsic information refinement process and the role of premature layers. Meanwhile, alignment- and fine-tuning-based methods are resource-intensive. In this paper, we propose PLI (Premature Layers Interpolation), a novel, training-free, and plug-and-play intervention designed to enhance factuality. PLI mitigates hallucinations by inserting premature layers formed through mathematical interpolation with adjacent layers. Inspired by stable diffusion and sampling steps, PLI extends the depth of information processing and transmission in LLMs, improving factual coherence. Experiments on four publicly available datasets demonstrate that PLI effectively reduces hallucinations while outperforming existing baselines in most cases. Further analysis suggests that the success of layer interpolation is closely linked to LLMs' internal mechanisms. Our dataset and code are available at https://github.com/CuSO4-Chen/PLI.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。