arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.12052cs.CL
收藏

小模型,大洞察:利用轻量代理模型决定何时以及为大语言模型检索什么

Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs

Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Baichuan Intelligent Technology(百川智能科技)

Jiejun Tan, Zhicheng Dou, Yutao Zhu, Peidong Guo, Kun Fang, Ji-Rong Wen

AI总结:

本文提出SlimPLM方法,利用轻量代理模型检测大语言模型缺失的知识,精准决定检索时机与内容,在降低推理成本的同时显著提升了问答任务的端到端性能。

摘要:

大语言模型(LLMs)与搜索引擎的结合代表了知识获取方法的重大演进。然而,确定LLM已经掌握的知识以及需要借助搜索引擎的知识,仍是一个未解决的问题。大多数现有方法通过LLM自身初步回答或推理的结果来解决此问题,但这带来了过高的计算成本。本文提出了一种新颖的协作方法,即SlimPLM,通过轻量代理模型检测LLM中缺失的知识,以增强LLM的知识获取过程。我们采用一个参数量远少的代理模型,将其答案作为启发式答案。然后利用启发式答案预测回答用户问题所需的知识,以及LLM内部已知和未知的知识。我们仅针对LLM不知道的问题中缺失的知识进行检索。在五个数据集和两个LLM上的大量实验结果表明,LLM在问答任务中的端到端性能显著提升,以更低的LLM推理成本达到或超越了当前最先进模型。

英文摘要:

The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑