SPAR:基于会话的适应性检索管道
SPAR: Session-based Pipeline for Adaptive Retrieval on Legacy File Systems
AI总结:
SPAR通过轻量级两阶段流程整合LLMs与RAG架构,提升遗留文件系统中检索效率与准确性。
摘要:
从历史数据中提取价值对于企业决策至关重要。然而,大量信息仍无法在缺乏结构化组织和语义索引的大型遗留文件系统中被访问,这使得检索和分析效率低下且容易出错。我们介绍了SPAR(基于会话的适应性检索管道),一种将大型语言模型(LLMs)整合到检索增强生成(RAG)架构中的概念框架,专门设计用于遗留企业环境。与传统RAG管道不同,后者需要昂贵地构建和维护完整规模的向量数据库,以镜像整个文件系统,而SPAR采用轻量级的两阶段过程:首先创建语义元数据索引,然后根据会话需求动态生成向量数据库。这种设计减少了计算开销,同时提高了检索的透明性、可控性和相关性。我们提供了比较SPAR与标准LLM基于RAG管道的理论复杂度分析,展示了其计算优势。为了验证该框架,我们将其应用于一个合成的企业级文件系统,其中包含大量生物医学文献,展示了在检索效果和下游模型准确性方面的改进。最后,我们讨论了设计权衡,并概述了在不同企业环境中部署SPAR面临的开放挑战。
英文摘要:
The ability to extract value from historical data is essential for enterprise decision-making. However, much of this information remains inaccessible within large legacy file systems that lack structured organization and semantic indexing, making retrieval and analysis inefficient and error-prone. We introduce SPAR (Session-based Pipeline for Adaptive Retrieval), a conceptual framework that integrates Large Language Models (LLMs) into a Retrieval-Augmented Generation (RAG) architecture specifically designed for legacy enterprise environments. Unlike conventional RAG pipelines, which require costly construction and maintenance of full-scale vector databases that mirror the entire file system, SPAR employs a lightweight two-stage process: a semantic Metadata Index is first created, after which session-specific vector databases are dynamically generated on demand. This design reduces computational overhead while improving transparency, controllability, and relevance in retrieval. We provide a theoretical complexity analysis comparing SPAR with standard LLM-based RAG pipelines, demonstrating its computational advantages. To validate the framework, we apply SPAR to a synthesized enterprise-scale file system containing a large corpus of biomedical literature, showing improvements in both retrieval effectiveness and downstream model accuracy. Finally, we discuss design trade-offs and outline open challenges for deploying SPAR across diverse enterprise settings.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。