揭示与缓解检索增强型大型语言模型中的检索器不一致性
Unraveling and Mitigating Retriever Inconsistencies in Retrieval-Augmented Large Language Models
- Harbin Institute of Technology（哈尔滨工业大学）
- XVERSE Technology Inc.（XVERSE科技公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本文揭示了检索增强型大语言模型中存在的检索器不一致性问题，通过理论分析退化行为成因，提出了可训练的检索器集成框架EoR，自适应检索多知识源以减少阅读器错误，显著提升了开放域问答性能。
AI中文摘要：
尽管检索增强型大型语言模型（RALMs）在事实性方面展现出优越性，但它们并未能持续超越原始的无检索语言模型（LMs）。我们的实验表明，这种样本级别的性能不一致性不仅存在于检索增强与无检索LM之间，也存在于不同检索器之间。为理解这一现象，我们研究了RALMs的退化行为，并在理论上将其分解为四类。基于该分解的进一步分析表明，知识源的固有差异和阅读器模型不可预测的退化是导致不一致性的主要原因。基于分析，我们提出了检索器集成（EoR），这是一个可训练框架，能够自适应地从不同知识源检索并有效减少不可预测的阅读器错误。我们在开放域问答上的实验表明，EoR通过大幅减少不一致行为，显著提升了相较于单一检索器RALM的性能。
英文摘要：
Although Retrieval-Augmented Large Language Models (RALMs) demonstrate their superiority in terms of factuality, they do not consistently outperform the original retrieval-free Language Models (LMs). Our experiments reveal that this example-level performance inconsistency exists not only between retrieval-augmented and retrieval-free LM but also among different retrievers. To understand this phenomenon, we investigate the degeneration behavior of RALMs and theoretically decompose it into four categories. Further analysis based on our decomposition reveals that the innate difference in knowledge sources and the unpredictable degeneration of the reader model contribute most to the inconsistency. Drawing from our analysis, we introduce Ensemble of Retrievers (EoR), a trainable framework that can adaptively retrieve from different knowledge sources and effectively decrease unpredictable reader errors. Our experiments on Open Domain Question Answering show that EoR substantially improves performance over the RALM with a single retriever by considerably reducing inconsistent behaviors.
