The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.
R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Beijing Normal University(北京师范大学)
AI Lab, Princeton University(普林斯顿大学人工智能实验室);University of Michigan(密歇根大学);Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University(清华大学交叉信息学院);Shanghai Jiao Tong University(上海交通大学);University of Edinburgh(爱丁堡大学);King’s College London(伦敦国王学院)
XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags
Ahsanullah University of Science & Technology(阿萨努拉科技大学);University of Alberta(阿尔伯塔大学);Université de Sherbrooke(谢布鲁克大学);Salesforce Research(Salesforce研究院);Nanyang Technological University(南洋理工大学)
Learning to Shop Like Humans: A Review-driven Retrieval-Augmented Recommendation Framework with LLMs
Chongqing University(重庆大学);Southwest Jiaotong University(西南交通大学);Hong Kong Generative AI Research and Development Center(香港生成式人工智能研究与发展中心);City University of Hong Kong(香港城市大学)
When Knowledge Is Not Free: Cost-Aware Evidence Selection in Retrieval-Augmented Generation
Northeastern University(东北大学);Technical University of Munich(慕尼黑技术大学);GESIS – Leibniz Institute for the Social Sciences(GESIS——莱比锡社会科学研究所);Technion–Israel Institute of Technology(技术学院——以色列理工学院);NVIDIA Research(NVIDIA研究);University of Groningen(格罗宁根大学)