arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.13296cs.IRcs.AI
收藏

PRIME:面向结构化文档检索的预检索索引建模增强方案

PRIME: Pre-Retrieval Index Modeling Enhancement for Structured Document Retrieval-Real

HKUST-GZ(香港科技大学(广州))

Cheuk Hei Chung, Li Liu

AI总结:

本文针对LLM智能体的外部知识库检索需求,提出PRIME索引优化框架,通过语义重构与元数据合成路径提升检索效率与Answer-F1,明确索引优化的适用场景且效果可被强重排序器抵消。

摘要:

高精度外部知识库检索对大语言模型(LLM)智能体的有效运行至关重要。然而,现有信息检索(IR)研究主要优化查询侧的检索器(如ColBERT、BGE)和生成器(如Self-RAG、GraphRAG),仅将语料库侧的表示简化为分块处理,导致索引侧质量这一关键优化方向未被充分探索。本文提出预检索索引建模与增强框架PRIME,该框架包含两个互补的优化路径:一是由LLM驱动的语义重构(SR)路径,将长文档转换为结构化、信息密度高的段落;二是确定性元数据合成(MS)与分层检索(HR)路径,提取语义锚点与标题结构以实现从粗到细的段落检索。评估结果显示,确定性元数据匹配了感知结构分块的检索质量,同时将嵌入计算量降低45.8倍,索引向量数量减少22.4倍;在此基础上,语义重构(SR)在全部署场景下进一步将Answer-F1指标提升13.7%,并呈现出可量化的质量-保真度-成本权衡关系。在QASPER、BRIGHT和NFCorpus三个数据集上,本文还明确了决定索引侧优化是提升还是降低检索效果的三种定性场景,同时证明在强重排序器作用下,索引侧的这些影响会被抵消。

英文摘要:

High-precision retrieval over external knowledge bases is essential for LLM agents to operate effectively. However, existing Information Retrieval (IR) research primarily optimizes query-side retrievers (e.g., ColBERT, BGE) and generators (e.g., Self-RAG, GraphRAG), while reducing corpus-side representation to chunking. This leaves index-side quality an underexplored lever. We propose Pre-Retrieval Index Modeling & Enhancement (PRIME), an index-side optimization framework built on two complementary tracks: an LLM-driven Semantic Restructuring (SR) track that transforms long documents into structured, information-dense sections, and a deterministic Metadata Synthesis (MS) and Hierarchical Retrieval (HR) track that extracts semantic anchors and heading structure for coarse-to-fine section retrieval. Our evaluations show that deterministic metadata matches structure-aware chunking's retrieval quality while cutting embedding computation by 45.8 $\times$ and index vector count by 22.4 $\times$. On top of these gains, Semantic Restructuring (SR) further improves Answer-F1 by 13.7% in full deployment, with characterized quality-fidelity-cost trade-offs. Across QASPER, BRIGHT, and NFCorpus, we further characterize three qualitative regimes that determine whether index-side optimization helps or degrades retrieval, while showing that these index-side effects are neutralized under a strong reranker.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑