Retrieval augmentation can aid language models (LMs) in knowledge-intensive tasks by supplying them with external information. Prior works on retrieval augmentation usually jointly fine-tune the retriever and the LM, making them closely coupled. In this paper, we explore the scheme of generic retrieval plug-in: the retriever is to assist target LMs that may not be known beforehand or are unable to be fine-tuned together. To retrieve useful documents for unseen target LMs, we propose augmentation-adapted retriever (AAR), which learns LM's preferences obtained from a known source LM. Experiments on the MMLU and PopQA datasets demonstrate that our AAR trained with a small source LM is able to significantly improve the zero-shot generalization of larger target LMs ranging from 250M Flan-T5 to 175B InstructGPT. Further analysis indicates that the preferences of different LMs overlap, enabling AAR trained with a single source LM to serve as a generic plug-in for various target LMs. Our code is open-sourced at https://github.com/OpenMatch/Augmentation-Adapted-Retriever.
C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学人工智能学院);Tongyi Lab(通义实验室);Soochow University(苏州大学);University of Oxford(牛津大学);Tsinghua University(清华大学)
Improve Dense Passage Retrieval with Entailment Tuning
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Thrust of Artificial Intelligence(人工智能分支(或:人工智能 thrust,按原文直译))
MemAdapter: Fast Alignment across Agent Memory Paradigms via Generative Subgraph Retrieval
The University of Manchester(曼彻斯特大学);Stanford University(斯坦福大学);Imperial College London(伦敦帝国理工学院);National Institute of Advanced Industrial Science(国家先进工业科学与技术研究院)
Automated Regulatory Compliance Question Answering in Financial Services with Domain-Adapted Retrieval-Augmented Generation
University of Bonn(波恩大学);Lamarr-Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所);Universitat Rovira i Virgili(罗维拉-维尔吉利大学);Fraunhofer IAIS(弗劳恩霍夫智能分析和信息系统研究所)
Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models
LUMIA Lab(LUMIA实验室);School of Artificial Intelligence(人工智能学院);Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学);SJTU Paris Elite Institute of Technology(上海交通大学巴黎精英技术学院)
Plug-and-Play Knowledge Injection for Pre-trained Language Models
Renmin University of China(中国人民大学);Tsinghua University(清华大学);International Innovation Center of Tsinghua University(清华大学国际创新中心);Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));Tencent Inc(腾讯公司);Quan Cheng Laboratory(泉城实验室)
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags
Ahsanullah University of Science & Technology(阿萨努拉科技大学);University of Alberta(阿尔伯塔大学);Université de Sherbrooke(谢布鲁克大学);Salesforce Research(Salesforce研究院);Nanyang Technological University(南洋理工大学)