大型搜索模型:在 LLMs 时代重新定义搜索栈
Large Search Model: Redefining Search Stack in the Era of LLMs
AI总结:
本文提出大型搜索模型框架,利用单一大型语言模型(LLM)将查询理解、检索、排序等传统独立搜索组件统一为自回归文本生成任务,通过自然语言提示定制任务,旨在提升搜索质量并简化搜索栈,并通过概念验证实验探讨其实际应用挑战。
摘要:
现代搜索引擎建立在由不同组件构成的搜索栈之上,包括查询理解、检索、多阶段排序和问答等。这些组件通常被独立优化和部署。本文提出了一种名为 large search model 的新颖概念框架,通过用单一大型语言模型(LLM)统一搜索任务来重新定义传统搜索栈。所有任务均被表述为自回归文本生成问题,从而允许通过使用自然语言提示来定制任务。该框架利用 LLMs 强大的语言理解和推理能力,有望提升搜索结果质量,同时简化现有繁琐的搜索栈。为验证该框架的可行性,我们进行了一系列概念验证实验,并讨论了在真实搜索系统中实施该方法可能面临的挑战。
英文摘要:
Modern search engines are built on a stack of different components, including query understanding, retrieval, multi-stage ranking, and question answering, among others. These components are often optimized and deployed independently. In this paper, we introduce a novel conceptual framework called large search model, which redefines the conventional search stack by unifying search tasks with one large language model (LLM). All tasks are formulated as autoregressive text generation problems, allowing for the customization of tasks through the use of natural language prompts. This proposed framework capitalizes on the strong language understanding and reasoning capabilities of LLMs, offering the potential to enhance search result quality while simultaneously simplifying the existing cumbersome search stack. To substantiate the feasibility of this framework, we present a series of proof-of-concept experiments and discuss the potential challenges associated with implementing this approach within real-world search systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。