PluriHopRAG: 通过语料库特定文档结构学习实现全面、召回敏感的问答
PluriHopRAG: Exhaustive, Recall-Sensitive QA Through Corpus-Specific Document Structure Learning
AI总结：
本文提出PluriHopRAG，通过学习语料库特定文档结构分解查询，改进多跳问答任务的召回敏感性和全面性，在PluriHopWIND和Loong基准上取得显著提升。
摘要：
检索增强生成（RAG）已被用于问答系统，以在相关信息存在于单跳或多跳段落时提高性能。然而，许多现实场景（例如处理财务、法律、医疗报告）需要检查所有文档以查找相关信息，而没有明确的停止条件。我们称这些为多跳问题，并通过召回敏感性、全面性和精确性三个条件进行形式化。为研究此设置，我们引入PluriHopWIND，一个包含191个真实风产业报告的多语言诊断基准，包含48个多跳问题，具有高度重复性以反映现实数据集中的干扰挑战。朴素、基于图的和多模态RAG方法在PluriHopWIND上仅能达到40%的陈述级F1。受此启发，我们提出PluriHopRAG，通过合成示例学习分解查询，并在文档层面使用交叉编码器过滤器以最小化LLM推理的成本。我们在PluriHopWIND和基于财务、法律和科学报告的Loong基准上测试PluriHopRAG。在PluriHopWIND上，我们的方法在基础LLM上实现了18-52%的F1分数提升，而在Loong上，我们展示了比长上下文推理高出33%、比朴素RAG高出52%的改进。
英文摘要：
Retrieval-Augmented Generation (RAG) has been used in question answering (QA) systems to improve performance when relevant information is in one (single-hop) or multiple (multi-hop) passages. However, many real life scenarios (e.g. dealing with financial, legal, medical reports) require checking all documents for relevant information without a clear stopping condition. We term these pluri-hop questions, and formalize them by 3 conditions - recall sensitivity, exhaustiveness, and exactness. To study this setting, we introduce PluriHopWIND, a multilingual diagnostic benchmark of 48 pluri-hop questions over 191 real wind-industry reports, with high repetitiveness to reflect the challenge of distractors in real-world datasets. Naive, graph-based, and multimodal RAG methods only reach up to 40% statement-wise F1 on PluriHopWIND. Motivated by this, we propose PluriHopRAG, which learns from synthetic examples to decompose queries according to corpus-specific document structure, and employs a cross-encoder filter at the document level to minimize costly LLM reasoning. We test PluriHopRAG on PluriHopWIND and the Loong benchmark built on financial, legal and scientific reports. On PluriHopWIND, our method shows 18-52% F1 score improvement across base LLMs, while on Loong, we show 33% improvement over long-context reasoning and 52% improvement over naive RAG.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
