arXiv 2504.135342025-09-10cs.CLcs.AI

CoT-RAG：融合思维链与检索增强生成以提升大语言模型推理能力

CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China（华中科技大学计算机科学与技术学院）；Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China（华中科技大学光电实验室）；Department of Computer Science, Bowling Green State University, Ohio, USA（Bowling Green State University 计算机科学系）；Department of Computer Science, Aalborg University, Aalborg, Denmark（Aalborg University 计算机科学系）；Huawei Technologies Co., Ltd, Shenzhen, China（华为技术有限公司）

Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Weihao Wang, Xin Zhang, Yongjian Cui

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

提出CoT-RAG框架，结合知识图谱驱动的思维链生成、可学习案例检索增强和伪程序提示执行，在九个数据集上显著提升LLM推理准确率4.0%-44.3%。

摘要：

思维链（CoT）推理能提升大语言模型（LLMs）在复杂任务上的表现，但面临两个关键局限：仅依赖LLM生成的推理链时缺乏可靠性，以及自然语言提示相比代码提示的推理性能较低。为解决这些问题，我们提出CoT-RAG，一个包含三项关键设计的新型推理框架：（i）知识图谱驱动的CoT生成，利用知识图谱调节LLM的推理链生成，从而增强推理可信度；（ii）可学习知识案例感知的RAG，将检索增强生成（RAG）融入知识图谱以检索相关子案例和子描述，为LLM提供可学习信息；（iii）伪程序提示执行，通过引导LLM将推理任务作为伪程序执行来促进更强的逻辑严谨性。在涵盖三个推理任务的九个公开数据集上的评估显示，相比最先进方法，准确率显著提升，范围从4.0%到44.3%。此外，在四个领域特定数据集上的测试展示了卓越的准确性和高效执行，凸显了其实用适用性和可扩展性。我们的代码和数据可在https://github.com/hustlfy123/CoT-RAG获取。

英文摘要：

Chain-of-thought (CoT) reasoning boosts large language models' (LLMs) performance on complex tasks but faces two key limitations: a lack of reliability when solely relying on LLM-generated reasoning chains and lower reasoning performance from natural language prompts compared with code prompts. To address these issues, we propose CoT-RAG, a novel reasoning framework with three key designs: (i) Knowledge Graph-driven CoT Generation, featuring knowledge graphs to modulate reasoning chain generation of LLMs, thereby enhancing reasoning credibility; (ii) Learnable Knowledge Case-aware RAG, which incorporates retrieval-augmented generation (RAG) into knowledge graphs to retrieve relevant sub-cases and sub-descriptions, providing LLMs with learnable information; (iii) Pseudo Program Prompting Execution, which promotes greater logical rigor by guiding LLMs to execute reasoning tasks as pseudo-programs. Evaluations on nine public datasets spanning three reasoning tasks reveal significant accuracy gains-ranging from 4.0% to 44.3%-over state-of-the-art methods. Furthermore, tests on four domain-specific datasets demonstrate exceptional accuracy and efficient execution, underscoring its practical applicability and scalability. Our code and data are available at https: //github.com/hustlfy123/CoT-RAG.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。