STAIR（结构感知信息检索器）：用于文档结构增强的新型数据集与基于大语言模型的检索器

中文摘要 AI 辅助

检索增强生成（RAG）是利用大语言模型（LLM）生成准确、无幻觉答案的关键组件。LLM处理长上下文的能力不断提升，但仍存在“中间丢失”问题，因此精确的检索至关重要。当前的检索器将长上下文按长度拆分为可管理的块，此过程会丢失语料库中丰富且有信息量的语义全局结构。我们提出一种新型检索系统STA IR，它能让LLM利用语料库中的全局结构（如目录（ToC）），高效地从其模型参数中存储和检索信息。我们对微调后的可微搜索索引（DSI）系统进行了全面且细致的 ablation 研究，结果显示，目录有助于构建低幻觉（低于0.05%）的生成式信息检索（IR）系统，且能在训练样本极少的示例上实现泛化。为推动基于目录的检索这一新方向的研究，我们发布了SearchTome——一个由6个不同领域的18本书构建的多样化基准。STA IR在SearchTome上的Recall@1得分达到82.6%，而DSI为76.9%，差异具有统计学意义。STA IR轻松击败了其他强基线，如BM25（59.5%）、DPR（68.7%）和开箱即用的Mistral（13.8%）。

英文摘要

Retrieval Augmented Generation (RAG) is a key component for generating accurate and hallucination free answers using Large Language Models (LLMs). LLMs are improving at handling long context, but still suffer from "lost in the middle" problem. Thus, precise and accurate retrieval is important. Current retrievers chunk long context into length-based manageable chunks - in the process throwing away rich and informative semantic global structure in the corpus. We introduce a novel retrieval system STAIR that empowers an LLM to exploit global structure in a corpus such as a Table of Contents (ToC) to efficiently store and retrieve information from its model parameters. Our thorough and careful ablation studies with a finetuned Differentiable Search Index (DSI) system show that ToC helps build a low hallucination (less than 0.05%) generative Information Retrieval (IR) system and can generalize to examples where very few training samples are available. To further research in this novel direction of ToC based retrieval we release SearchTome - a diverse benchmark created from 18 books across 6 diverse domains to further research in this novel direction. STAIR achieves a high Recall@1 score of 82.6% on SearchTome as compared to DSI (76.9%), where the difference is found to be statistically significant. STAIR easily beats other strong baselines such as BM25 (59.5%), DPR (68.7%) and out-of-the-box Mistral (13.8%).

发表机构

• IBM（国际商业机器公司）
• Amazon Books Science（亚马逊图书科学部门）

机构由 AI 辅助整理，请以论文原文为准。