Knowledge Integration Decay in Search-Augmented Reasoning of Large Language Models
Department of Electrical and Computer Engineering, Seoul National University, Seoul, Korea(电气与计算机工程系,首尔国立大学,韩国首尔);Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Korea(人工智能交叉学科项目,首尔国立大学,韩国首尔)
Modern Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks by employing search-augmented reasoning to incorporate external knowledge into long chains of thought. However, we identify a critical yet underexplored bottleneck in this paradigm, termed Knowledge Integration Decay (KID). Specifically, we observe that as the length of reasoning generated before search grows, models increasingly fail to integrate retrieved evidence into subsequent reasoning steps, limiting performance even when relevant information is available. To address this, we propose Self-Anchored Knowledge Encoding (SAKE), a training-free inference-time strategy designed to stabilize knowledge utilization. By anchoring retrieved knowledge at both the beginning and end of the reasoning process, SAKE prevents it from being overshadowed by prior context, thereby preserving its semantic integrity. Extensive experiments on multi-hop QA and complex reasoning benchmarks demonstrate that SAKE significantly mitigates KID and improves performance, offering a lightweight yet effective solution for knowledge integration in agentic LLMs.
ChatCoT: Tool-Augmented Chain-of-Thought Reasoning on Chat-based Large Language Models
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);School of Information, Renmin University of China(中国人民大学信息学院);Beijing Key Laboratory of Big Data Management and Analysis Methods(北京大数据管理与分析方法重点实验室)
Fast, Slow, and Tool-augmented Thinking for LLMs: A Review
School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China(上海交通大学机械工程学院);School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China(上海交通大学计算机科学学院);Huawei Technologies Co., Ltd., Beijing 100084, China(华为技术有限公司);Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Hong Kong 999077, China(香港中文大学系统工程与工程管理系);Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong 999077, China(香港中文大学计算机科学与工程系);Huawei Hong Kong Research Center, Hong Kong 999077, China(华为香港研究中心)
Physics Reasoner: Knowledge-Augmented Reasoning for Solving Physics Problems with Large Language Models
Institute for Artificial Intelligence, Tsinghua University (THUAI)(清华大学人工智能研究院);Beijing National Research Center for Information Science and Technology (BNRist)(北京信息科学与技术国家研究中心);Department of Automation, Tsinghua University(清华大学自动化系);TMLR Group, Hong Kong Baptist University(香港浸会大学TMLR组)
ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding
Kuaishou Technology Co., Ltd.(快手科技有限公司);Institute for Clarity in Documentation(文献清晰度研究所);Inria Paris-Rocquencourt(法国国家信息与自动化研究所巴黎-罗康分所);Rajiv Gandhi University(拉吉夫甘地大学);Tsinghua University(清华大学);Palmer Research Laboratories(帕尔默研究实验室)
Improving Context Fidelity via Native Retrieval-Augmented Reasoning
DIRO, Université de Montréal(蒙特利尔大学计算机科学与运筹学系);MetaGPT;Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);McGill University(麦吉尔大学);Yale University(耶鲁大学);Canada CIFAR AI Chair(加拿大CIFAR人工智能教席)
RAG-Star: Enhancing Deliberative Reasoning with Retrieval Augmented Verification and Refinement
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Wuhan University of Science and Technology(武汉科技大学);National University of Singapore(新加坡国立大学);Northeastern University at Qinhuangdao(东北大学秦皇岛分校);BOSS Zhipin(BOSS直聘)
ARise: Towards Knowledge-Augmented Reasoning via Risk-Adaptive Search
Shanghai AI Laboratory(上海人工智能实验室);Shanghai Innovation Institute(上海创新研究院);Shanghai Jiao Tong University(上海交通大学);SenseTime(商汤科技);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);Tongji University(同济大学);Hangzhou Institute for Advanced Study(杭州高等研究院);University of Chinese Academy of Sciences(中国科学院大学)
SciAgent: Tool-augmented Language Models for Scientific Reasoning
Microsoft(微软公司);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Tsinghua University(清华大学);Singapore Management University(新加坡管理大学);Nanyang Technological University(南洋理工大学)
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought
Speechmatics, Apollo Research(Speechmatics与Apollo研究机构);University of Oxford(牛津大学);NYU, Anthropic(纽约大学与Anthropic);NYU(纽约大学);Anthropic, NYU(Anthropic与纽约大学)
Eliciting Critical Reasoning in Retrieval-Augmented Language Models via Contrastive Explanations
Idiap Research Institute(Idiap研究机构);Department of Computer Science, University of Manchester(曼彻斯特大学计算机科学系);National Biomarker Centre (NBC), CRUK Manchester Institute(国家生物标志物中心(NBC),CRUK曼彻斯特研究所)
CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China(华中科技大学计算机科学与技术学院);Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China(华中科技大学光电实验室);Department of Computer Science, Bowling Green State University, Ohio, USA(Bowling Green State University 计算机科学系);Department of Computer Science, Aalborg University, Aalborg, Denmark(Aalborg University 计算机科学系);Huawei Technologies Co., Ltd, Shenzhen, China(华为技术有限公司)
Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solving
AAII, University of Technology Sydney, New South Wales, Australia(AAII,悉尼大学,新南威尔士州,澳大利亚);Eindhoven University of Technology, Eindhoven, The Netherlands(埃因霍温理工大学,埃因霍温,荷兰);University of Liverpool, Liverpool, United Kingdom(利物浦大学,利物浦,英国);University of New South Wales, New South Wales, Australia(新南威尔士大学,新南威尔士州,澳大利亚)
RAGAR, Your Falsehood Radar: RAG-Augmented Reasoning for Political Fact-Checking using Multimodal Large Language Models
Institute for Natural Language Processing, University of Stuttgart(斯图加特大学自然语言处理研究所);appliedAI Institute for Europe gGmbH(欧洲appliedAI研究所(非营利有限责任公司));appliedAI Initiative GmbH(appliedAI倡议有限责任公司)