HIRAG：层级思维指令微调检索增强生成
HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation
AI总结：
针对RAG生成模型缺乏细粒度任务关注和思维链利用的问题，提出HIRAG方法，通过层级思维指令微调赋予模型过滤、组合和RAG特定推理能力，显著提升多个基准数据集性能。
摘要：
检索增强生成（RAG）已成为应对大型语言模型在处理实时信息和领域特定问题时所面临挑战的基本范式。传统RAG系统主要依赖大型语言模型自身的上下文学习（ICL）能力，但对RAG生成模型所需的具体能力缺乏深入研究，导致文档质量不一致和检索系统不完善等问题。即使是少数对RAG生成模型进行微调的研究，也常常缺乏对RAG任务的细粒度关注，或未能更深入地利用思维链过程。为解决这一问题，我们提出RAG模型应具备三种递进式层级能力：（1）过滤：选择相关信息的能力；（2）组合：跨段落组合语义信息的能力；（3）RAG特定推理：利用内部知识进一步处理外部知识的能力。因此，我们引入了新的RAG指令微调方法——层级思维指令微调检索增强生成（HIRAG），该方法融入了“先思考后回答”的策略。该方法通过利用多层级渐进式思维链，增强了模型的开放书考试能力。实验表明，HIRAG训练策略显著提升了模型在RGB、PopQA、MuSiQue、HotpotQA和PubmedQA等数据集上的性能。
英文摘要：
Retrieval-augmented generation (RAG) has become a fundamental paradigm for addressing the challenges faced by large language models in handling real-time information and domain-specific problems. Traditional RAG systems primarily rely on the in-context learning (ICL) capabilities of the large language model itself. Still, in-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to challenges with inconsistent document quality and retrieval system imperfections. Even the limited studies that fine-tune RAG generative models often \textit{lack a granular focus on RAG task} or \textit{a deeper utilization of chain-of-thought processes}. To address this, we propose that RAG models should possess three progressively hierarchical abilities (1) Filtering: the ability to select relevant information; (2) Combination: the ability to combine semantic information across paragraphs; and (3) RAG-specific reasoning: the ability to further process external knowledge using internal knowledge. Thus, we introduce our new RAG instruction fine-tuning method, Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (HIRAG) incorporates a "think before answering" strategy. This method enhances the model's open-book examination capability by utilizing multi-level progressive chain-of-thought. Experiments show that the HIRAG training strategy significantly improves the model's performance on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
