Improving Zero-shot LLM Re-Ranker with Risk Minimization
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院)
Xiaowei Yuan, Zhao Yang, Yequan Wang, Jun Zhao, Kang Liu
In the Retrieval-Augmented Generation (RAG) system, advanced Large Language Models (LLMs) have emerged as effective Query Likelihood Models (QLMs) in an unsupervised way, which re-rank documents based on the probability of generating the query given the content of a document. However, directly prompting LLMs to approximate QLMs inherently is biased, where the estimated distribution might diverge from the actual document-specific distribution. In this study, we introduce a novel framework, $\mathrm{UR^3}$, which leverages Bayesian decision theory to both quantify and mitigate this estimation bias. Specifically, $\mathrm{UR^3}$ reformulates the problem as maximizing the probability of document generation, thereby harmonizing the optimization of query and document generation probabilities under a unified risk minimization objective. Our empirical results indicate that $\mathrm{UR^3}$ significantly enhances re-ranking, particularly in improving the Top-1 accuracy. It benefits the QA tasks by achieving higher accuracy with fewer input documents.
Large Language Models are Zero-Shot Rankers for Recommender Systems
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);UC San Diego(加州大学圣迭戈分校);School of Information, Renmin University of China(中国人民大学信息资源管理学院);WeChat, Tencent(腾讯微信)
Improving Passage Retrieval with Zero-Shot Question Generation
McGill University(麦吉尔大学);Mila - Quebec AI Institute(米拉-魁北克人工智能研究所);Meta AI Research(Meta人工智能研究院);Google Research(谷歌研究院);University of Washington(华盛顿大学)
MedCPT: Contrastive Pre-trained Transformers with Large-scale PubMed Search Logs for Zero-shot Biomedical Information Retrieval
National Center for Biotechnology Information (NCBI)(国家生物技术信息中心);National Library of Medicine (NLM)(国家医学图书馆);National Institutes of Health (NIH)(国立卫生研究院)
MedCPT 利用 2.55 亿条 PubMed 点击日志进行对比预训练,首次实现生物医学零样本语义检索,并在六个 IR 任务上超越 GPT-3 规模基线模型。