RetrievalQA：评估自适应检索增强生成在简短开放式问答中的表现
RetrievalQA: Assessing Adaptive Retrieval-Augmented Generation for Short-form Open-Domain Question Answering
AI总结：
本文提出RetrievalQA基准评估自适应检索增强生成，并发现现有方法依赖阈值或提示不足，进而提出无需校准与训练的TA-ARE方法以有效判断检索必要性。
摘要：
自适应检索增强生成（ARAG）旨在动态决定查询是否需要检索，而非盲目检索，以提升所获信息的效率与相关性。然而，以往研究很大程度上忽视了对ARAG方法的评估，导致其有效性未被充分研究。本文提出了一个基准RetrievalQA，包含1,271个简短形式的问题，涵盖新世界和长尾知识。回答这些问题所需的知识在LLMs中不存在；因此，必须检索外部信息才能正确作答。这使得RetrievalQA成为评估现有ARAG方法的合适测试平台。我们观察到，基于校准的方法严重依赖阈值调整，而简单的提示方法不足以引导LLMs做出可靠的检索决策。基于我们的发现，我们提出了时间感知自适应检索（TA-ARE），这是一种简单而有效的方法，可帮助LLMs在无需校准或额外训练的情况下评估检索的必要性。数据集和代码将发布于https://github.com/hyintell/RetrievalQA
英文摘要：
Adaptive retrieval-augmented generation (ARAG) aims to dynamically determine the necessity of retrieval for queries instead of retrieving indiscriminately to enhance the efficiency and relevance of the sourced information. However, previous works largely overlook the evaluation of ARAG approaches, leading to their effectiveness being understudied. This work presents a benchmark, RetrievalQA, comprising 1,271 short-form questions covering new world and long-tail knowledge. The knowledge necessary to answer the questions is absent from LLMs; therefore, external information must be retrieved to answer correctly. This makes RetrievalQA a suitable testbed to evaluate existing ARAG methods. We observe that calibration-based methods heavily rely on threshold tuning, while vanilla prompting is inadequate for guiding LLMs to make reliable retrieval decisions. Based on our findings, we propose Time-Aware Adaptive Retrieval (TA-ARE), a simple yet effective method that helps LLMs assess the necessity of retrieval without calibration or additional training. The dataset and code will be available at https://github.com/hyintell/RetrievalQA
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
