MoM：面向检索增强生成系统的场景感知文档记忆混合模型
MoM: Mixtures of Scenario-Aware Document Memories for Retrieval-Augmented Generation Systems
AI总结：
针对RAG被动分块限制知识内化的问题，提出MoM框架，通过LLM模拟专家生成大纲、多路径采样评估及反向推理训练SLM，构建场景感知文档记忆，经三层检索机制在三个领域验证，提升语义完整性与智能处理能力。
摘要：
传统的检索增强生成（RAG）范式通常针对接收到的查询理解相关文本块，这从根本上限制了知识内化的深度和推理能力。为解决这一局限，我们的研究将RAG中的文本处理从被动分块转变为主动理解，将此过程定义为文档记忆提取，旨在模拟人类阅读时的认知过程。在此基础上，我们提出了场景感知文档记忆混合模型（MoM）框架，该框架旨在高效处理来自多个领域的文档，并训练小型语言模型（SLMs）获得主动探索和构建文档记忆的能力。MoM首先指导大型语言模型（LLMs）模拟领域专家生成文档逻辑大纲，从而引导结构化分块和核心内容提取。它采用多路径采样和多视角评估机制，专门设计了代表分块清晰度和提取完整性的综合指标，以选择最优的文档记忆。此外，为了在SLMs训练中注入更深层次的人类阅读能力，我们引入了反向推理策略，该策略从高质量结果中推导出精炼的专家思维路径。最后，利用MoM生成的多种形式内容，我们开发了一种三层文档记忆检索机制，该机制基于我们从概率建模角度进行的理论证明。跨三个不同领域的广泛实验结果表明，MoM框架不仅解决了现有RAG系统中的文本分块难题，为LLMs提供了语义完整的文档记忆，还为SLMs实现以人为中心的智能文本处理铺平了道路。
英文摘要：
The traditional RAG paradigm, which typically engages in the comprehension of relevant text chunks in response to received queries, inherently restricts both the depth of knowledge internalization and reasoning capabilities. To address this limitation, our research transforms the text processing in RAG from passive chunking to proactive understanding, defining this process as document memory extraction with the objective of simulating human cognitive processes during reading. Building upon this, we propose the Mixtures of scenario-aware document Memories (MoM) framework, engineered to efficiently handle documents from multiple domains and train small language models (SLMs) to acquire the ability to proactively explore and construct document memories. The MoM initially instructs large language models (LLMs) to simulate domain experts in generating document logical outlines, thereby directing structured chunking and core content extraction. It employs a multi-path sampling and multi-perspective evaluation mechanism, specifically designing comprehensive metrics that represent chunk clarity and extraction completeness to select the optimal document memories. Additionally, to infuse deeper human-like reading abilities during the training of SLMs, we incorporate a reverse reasoning strategy, which deduces refined expert thinking paths from high-quality outcomes. Finally, leveraging diverse forms of content generated by MoM, we develop a three-layer document memory retrieval mechanism, which is grounded in our theoretical proof from the perspective of probabilistic modeling. Extensive experimental results across three distinct domains demonstrate that the MoM framework not only resolves text chunking challenges in existing RAG systems, providing LLMs with semantically complete document memories, but also paves the way for SLMs to achieve human-centric intelligent text processing.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
