发表机构
Université du Québec à Trois-Rivières（魁北克大学三河城分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对RAG检索多样化效果争议，提出基于候选池冗余度和查询证据需求的查询自适应多样化规则及RNG-Score重排序器，实现按需多样化以提升检索与答案质量。
AI 中文摘要
检索多样化在检索增强生成（RAG）框架中广泛可用，然而先前的研究对其是否能提高检索和答案质量存在分歧。我们表明，其有效性主要随候选池冗余度变化，其模式与查询所需的不同证据片段数量一致。通过受控的近重复注入和生产风格的重叠分块，我们发现多样化在干净池上会损害相关性、证据覆盖率和答案质量，但在多证据任务中，当冗余导致最近邻检索选择重复段落时，多样化则变得有益。因此，我们引入了一种查询自适应规则，仅当最近邻top-$k$选择中有效不同文档数量低于查询的证据需求时，才进行多样化。该规则从现有嵌入中计算得出，能捕获大部分可实现的增益，跨数据集和编码器迁移，并自动简化为单证据查询的最近邻检索。我们还引入了RNG-Score，一种具有精确最近邻回退的几何重排序器，其边界指示重复结构。总体而言，我们得出结论，多样化应根据可观察的冗余度和证据需求有选择地使用。
英文摘要
Retrieval diversification is widely available in retrieval-augmented generation (RAG) frameworks, yet prior studies disagree on whether it improves retrieval and answer quality. We show that its effectiveness varies primarily with candidate-pool redundancy, in a pattern consistent with the number of distinct evidence pieces a query requires. Using controlled near-duplicate injection and production-style overlapping chunking, we find that diversification harms relevance, evidence coverage and answer quality on clean pools, but becomes beneficial on multi-evidence tasks when redundancy causes nearest-neighbor retrieval to select repeated passages. We therefore introduce a query-adaptive rule that diversifies only when the effective number of distinct documents in the nearest-neighbor top-$k$ selection falls below the query's evidence requirement. Computed from existing embeddings, the rule captures most of the achievable gain, transfers across datasets and encoders and automatically reduces to nearest-neighbor retrieval for single-evidence queries. We also introduce RNG-Score, a geometric reranker with an exact nearest-neighbor fallback whose margin indicates duplicate structure. Overall, we conclude that diversification should be used selectively, based on observable redundancy and evidence requirements.
Comments36 pages, 8 figures, 13 tables. Code and results: https://github.com/GuillaumeBrouillette/finding-the-right-balance
