arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09517cs.CL
收藏

在大语言模型的搜索增强推理中知识整合衰减

Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models

Department of Electrical and Computer Engineering, Seoul National University, Seoul, Korea(电气与计算机工程系,首尔国立大学,韩国首尔);Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea(人工智能交叉学科项目,首尔国立大学,韩国首尔)

Sangwon Yu, Ik-hwan Kim, Donghun Kang, Bongkyu Hwang, Junhwa Choi, Suk-hoon Jung, Seungki Hong, Taehee Lee, Sungroh Yoon

AI总结:

本研究提出SAKE方法,通过在推理过程前后锚定检索知识,缓解大语言模型在搜索增强推理中的知识整合衰减问题,提升多跳问答和复杂推理任务的性能。

摘要:

现代大语言模型(LLMs)通过采用搜索增强推理来将外部知识整合进长推理链中,在复杂任务中展示了显著的能力。然而,我们发现这一范式中存在一个关键但尚未被深入探讨的瓶颈,称为知识整合衰减(KID)。具体而言,我们观察到随着在搜索之前生成的推理长度增加,模型越来越难以将检索到的证据整合到后续的推理步骤中,即使有相关的信息可用,也会限制性能。为了解决这个问题,我们提出了Self-Anchored Knowledge Encoding(SAKE),一种无需训练的推理时策略,旨在稳定知识利用。通过在推理过程的开始和结束都锚定检索到的知识,SAKE防止其被先前上下文所掩盖,从而保持其语义完整性。在多跳问答和复杂推理基准上的广泛实验表明,SAKE显著减轻了KID并提高了性能,为代理LLM中的知识整合提供了一种轻量但有效的解决方案。

英文摘要:

Modern Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks by employing search-augmented reasoning to incorporate external knowledge into long chains of thought. However, we identify a critical yet underexplored bottleneck in this paradigm, termed Knowledge Integration Decay (KID). Specifically, we observe that as the length of reasoning generated before search grows, models increasingly fail to integrate retrieved evidence into subsequent reasoning steps, limiting performance even when relevant information is available. To address this, we propose Self-Anchored Knowledge Encoding (SAKE), a training-free inference-time strategy designed to stabilize knowledge utilization. By anchoring retrieved knowledge at both the beginning and end of the reasoning process, SAKE prevents it from being overshadowed by prior context, thereby preserving its semantic integrity. Extensive experiments on multi-hop QA and complex reasoning benchmarks demonstrate that SAKE significantly mitigates KID and improves performance, offering a lightweight yet effective solution for knowledge integration in agentic LLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑