无自我认知的自适应检索?让不确定性回归本位
Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home
AI总结:
针对现有自适应检索忽略效率评估及与不确定性技术对比的不足,研究对35种自适应检索方法在6个数据集上的综合分析,发现不确定性估计技术在保相当QA性能同时,效率和自我认知更优。
摘要:
检索增强生成(Retrieval Augmented Generation, RAG)可提升问答(Question Answering, QA)的准确性并解决大语言模型(Large Language Models, LLMs)的幻觉问题,但会大幅增加计算成本。此外,RAG并非始终必要,因为它可能引入无关信息。近期的自适应检索方法利用LLM自我认知将LLMs的内在知识与外部信息相结合,但这类方法往往忽略了效率评估,以及与不确定性估计技术的对比。我们填补了这一空白:针对35种自适应检索方法(包括8种最新方法和27种不确定性估计技术,在6个数据集上使用10项指标开展综合分析,评估维度涵盖QA性能、自我认知和效率。研究结果表明,不确定性估计技术在效率和自我认知方面往往优于复杂流水线,同时保持相当的QA性能。
英文摘要:
Retrieval Augmented Generation (RAG) improves correctness of Question Answering (QA) and addresses hallucinations in Large Language Models (LLMs), yet greatly increase computational costs. Besides, RAG is not always needed as may introduce irrelevant information. Recent adaptive retrieval methods integrate LLMs' intrinsic knowledge with external information appealing to LLM self-knowledge, but they often neglect efficiency evaluations and comparisons with uncertainty estimation techniques. We bridge this gap by conducting a comprehensive analysis of 35 adaptive retrieval methods, including 8 recent approaches and 27 uncertainty estimation techniques, across 6 datasets using 10 metrics for QA performance, self-knowledge, and efficiency. Our findings show that uncertainty estimation techniques often outperform complex pipelines in terms of efficiency and self-knowledge, while maintaining comparable QA performance.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。