arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.13331cs.CL
收藏

通过风险最小化改进零样本 LLM 重排器

Improving Zero-shot LLM Re-Ranker with Risk Minimization

Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)

Xiaowei Yuan, Zhao Yang, Yequan Wang, Jun Zhao, Kang Liu

AI总结:

本文提出 $\mathrm{UR^3}$ 框架,利用贝叶斯决策理论量化和缓解零样本 LLM 重排器近似查询似然模型时的估计偏差,通过统一风险最小化目标协调查询与文档生成概率的优化,显著提升了重排性能和 Top-1 准确率。

摘要:

在检索增强生成(RAG)系统中,先进的大型语言模型(LLMs)以无监督方式作为有效的查询似然模型(QLMs),根据给定文档内容生成查询的概率对文档进行重排。然而,直接提示 LLMs 近似 QLMs 本质上存在偏差,估计的分布可能与实际特定于文档的分布产生偏离。在本研究中,我们引入了一个新框架 $\mathrm{UR^3}$,利用贝叶斯决策理论来量化和缓解这种估计偏差。具体而言,$\mathrm{UR^3}$ 将问题重新表述为最大化文档生成概率,从而在统一的风险最小化目标下协调查询和文档生成概率的优化。我们的实证结果表明,$\mathrm{UR^3}$ 显著增强了重排效果,特别是在提高 Top-1 准确率方面。它通过以更少的输入文档实现更高的准确率,使 QA 任务受益。

英文摘要:

In the Retrieval-Augmented Generation (RAG) system, advanced Large Language Models (LLMs) have emerged as effective Query Likelihood Models (QLMs) in an unsupervised way, which re-rank documents based on the probability of generating the query given the content of a document. However, directly prompting LLMs to approximate QLMs inherently is biased, where the estimated distribution might diverge from the actual document-specific distribution. In this study, we introduce a novel framework, $\mathrm{UR^3}$, which leverages Bayesian decision theory to both quantify and mitigate this estimation bias. Specifically, $\mathrm{UR^3}$ reformulates the problem as maximizing the probability of document generation, thereby harmonizing the optimization of query and document generation probabilities under a unified risk minimization objective. Our empirical results indicate that $\mathrm{UR^3}$ significantly enhances re-ranking, particularly in improving the Top-1 accuracy. It benefits the QA tasks by achieving higher accuracy with fewer input documents.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑