ReSLLM:大语言模型是联邦搜索的强资源选择器
ReSLLM: Large Language Models are Strong Resource Selectors for Federated Search
AI总结:
针对联邦搜索中现有资源选择方法依赖人工标注、成本高昂的问题,提出ReSLLM方法,利用大语言模型实现零样本资源选择,并设计SLAT无监督微调方案进一步提升性能,实验验证了其有效性。
摘要:
联邦搜索涉及整合多个独立搜索引擎的结果,在为聊天机器人等基于大语言模型(LLM)的应用提供支持的检索增强生成(RAG)流程背景下,其重要性将日益凸显。这类系统通常会根据用户表述的性质,将查询分发到各类搜索引擎,涵盖专业型(如PubMed)和通用型(如Google)。联邦搜索的一个核心环节是资源选择——即在发出查询前挑选合适的资源,以确保高质量且快速的响应,同时控制调用外部搜索引擎的相关成本。然而,当前最先进(SOTA)的资源选择方法主要依赖基于特征的学习方法,这类方法往往需要为每个资源创建训练标签,过程既耗费人力又成本高昂。相比之下,大语言模型在自然语言处理(NLP)和信息检索(IR)任务中作为零样本方法已展现出强大的效能。我们假设,在联邦搜索场景下,大语言模型无需大量预定义标签或特征即可评估资源的相关性。本文提出了ReSLLM方法,该方法利用大语言模型在零样本设置下驱动联邦搜索中的资源选择。此外,我们设计了一种无监督微调方案——合成标签增强调优(Synthetic Label Augmentation Tuning,SLAT):利用现成的大语言模型预测历史记录中查询与资源片段的相关性,再将这些预测结果用于对ReSLLM进行资源选择任务的微调。我们的实证评估与分析详细阐述了影响大语言模型在该场景下效能的因素。结果彰显了ReSLLM在资源选择方面的优势:不仅在零样本设置下具备具有竞争力的效能,而且通过SLAT方案微调后性能还能获得大幅提升。
英文摘要:
Federated search, which involves integrating results from multiple independent search engines, will become increasingly pivotal in the context of Retrieval-Augmented Generation pipelines empowering LLM-based applications such as chatbots. These systems often distribute queries among various search engines, ranging from specialized (e.g., PubMed) to general (e.g., Google), based on the nature of user utterances. A critical aspect of federated search is resource selection - the selection of appropriate resources prior to issuing the query to ensure high-quality and rapid responses, and contain costs associated with calling the external search engines. However, current SOTA resource selection methodologies primarily rely on feature-based learning approaches. These methods often involve the labour intensive and expensive creation of training labels for each resource. In contrast, LLMs have exhibited strong effectiveness as zero-shot methods across NLP and IR tasks. We hypothesise that in the context of federated search LLMs can assess the relevance of resources without the need for extensive predefined labels or features. In this paper, we propose ReSLLM. Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting. In addition, we devise an unsupervised fine tuning protocol, the Synthetic Label Augmentation Tuning (SLAT), where the relevance of previously logged queries and snippets from resources is predicted using an off-the-shelf LLM and then in turn used to fine-tune ReSLLM with respect to resource selection. Our empirical evaluation and analysis details the factors influencing the effectiveness of LLMs in this context. The results showcase the merits of ReSLLM for resource selection: not only competitive effectiveness in the zero-shot setting, but also obtaining large when fine-tuned using SLAT-protocol.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。