arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.09402cs.CL
收藏

SEEK: 通过内部推理草图引导LLM推理用于RAG

SEEK: Steering LLM Reasoning for RAG via Internal Reasoning Sketches

School of Computer Science and Engineering, Northeastern University, China(东北大学计算机科学与工程学院);Department of Computer Science and Technology, Institute for AI, Tsinghua University, China(清华大学计算机科学与技术系,人工智能研究院);School of Intelligent Science and Technology, Nanjing University, China(南京大学智能科学与技术学院)

Xinze Li, Yuqing Lan, Zhenghao Liu, Haidong Xin, Yukun Yan, Shuo Wang, Zheni Zeng, Sen Mei, Ge Yu, Maosong Sun

AI总结:

提出SEEK框架,通过构建结构化引导草图,迭代检索和填充知识槽,减少冗余检索,提升RAG性能。

摘要:

检索增强生成(RAG)通过将外部知识融入生成过程来增强大型语言模型(LLM)。借助LLM的推理能力,现有方法利用这种能力实现迭代知识获取和积累,从而更好地支持答案生成。然而,随着推理轨迹的增长,积累的知识和先前生成的查询可能会干扰后续检索决策,导致子查询意图重复和知识获取冗余。为了解决这个问题,我们提出了SEEK,一种用于RAG的草图引导知识获取框架。SEEK首先提示LLM为给定问题构建一个结构化的引导草图。它由多组引导要点组成,每个要点后跟一个用于知识填充的槽位。在这些引导要点的指导下,SEEK迭代地检索和精炼知识,并填充相应的槽位以完成草图。然后,完成的草图作为上下文输入用于最终答案生成。实验结果表明,SEEK在多个任务上取得了比基线模型更好的性能。进一步分析表明,SEEK可以生成更多样化的子查询,减少冗余检索,并在外部知识利用和内部知识冲突缓解之间实现更好的平衡。所有代码可在 https://github.com/OpenBMB/PAGER 获取。

英文摘要:

Retrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by incorporating external knowledge into the generation process. Benefiting from the reasoning capabilities of LLMs, existing methods have leveraged such capabilities to enable iterative knowledge acquisition and accumulation, thereby better supporting answer generation. However, as the reasoning trajectory grows, the accumulated knowledge and previously generated queries may interfere with subsequent retrieval decisions, resulting in sub-queries with repetitive intents and redundant knowledge acquisition. To address this issue, we propose SEEK, a sketch-guided knowledge acquisition framework for RAG. SEEK first prompts the LLM to construct a structured steering sketch for the given question. It consists of multiple groups of steering gists, with each gist followed by a slot for knowledge filling. Guided by these steering gists, SEEK iteratively retrieves and refines knowledge, and fills the corresponding slots to complete the sketch. The completed sketch is then used as contextual input for final answer generation. Experimental results show that SEEK achieves better performance than baseline models across multiple tasks. Further analyses demonstrate that SEEK can generate more diverse sub-queries, reduce redundant retrieval, and achieve a better balance between external knowledge utilization and internal knowledge conflict mitigation. All codes are available at https://github.com/OpenBMB/PAGER.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑