RankCoT：通过排序思维链精炼检索增强生成的知识
RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts
AI总结：
RankCoT通过融入重排序信号和自反思机制，在训练中生成并精炼思维链总结，以过滤检索文档中的噪声，提升RAG系统生成答案的准确性。
摘要：
检索增强生成（RAG）通过引入外部知识来提升大型语言模型（LLMs）的性能。然而，LLMs在有效利用检索文档中的知识方面仍面临挑战，常常被不相关或嘈杂的信息误导。为解决此问题，我们提出了RankCoT，一种知识精炼方法，它基于给定查询和所有检索文档，在生成基于思维链（CoT）的总结以精炼知识时融入重排序信号。在训练阶段，RankCoT促使LLM基于查询和单个文档生成思维链（CoT）候选，然后微调LLM，使其基于所有检索文档直接从这些候选输出中复现最佳CoT，这要求LLM在生成CoT式总结时过滤掉不相关的文档。此外，RankCoT引入了一种自我反思机制，进一步精炼CoT输出，从而产生更高质量的训练数据。我们的实验证明了RankCoT的有效性，显示其性能优于其他知识精炼模型。进一步分析表明，RankCoT能提供更短但有效的精炼结果，使生成器能产生更准确的答案。所有代码和数据可在https://github.com/NEUIR/RankCoT获取。
英文摘要：
Retrieval-Augmented Generation (RAG) enhances the performance of Large Language Models (LLMs) by incorporating external knowledge. However, LLMs still encounter challenges in effectively utilizing the knowledge from retrieved documents, often being misled by irrelevant or noisy information. To address this issue, we introduce RankCoT, a knowledge refinement method that incorporates reranking signals in generating CoT-based summarization for knowledge refinement based on given query and all retrieval documents. During training, RankCoT prompts the LLM to generate Chain-of-Thought (CoT) candidates based on the query and individual documents. It then fine-tunes the LLM to directly reproduce the best CoT from these candidate outputs based on all retrieved documents, which requires LLM to filter out irrelevant documents during generating CoT-style summarization. Additionally, RankCoT incorporates a self-reflection mechanism that further refines the CoT outputs, resulting in higher-quality training data. Our experiments demonstrate the effectiveness of RankCoT, showing its superior performance over other knowledge refinement models. Further analysis reveals that RankCoT can provide shorter but effective refinement results, enabling the generator to produce more accurate answers. All code and data are available at https://github.com/NEUIR/RankCoT.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
