arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2510.18527cs.IR
收藏

作为稀疏检索器的大语言模型:第一阶段商品搜索框架

LLMs as Sparse Retrievers:A Framework for First-Stage Product Search

State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所人工智能安全重点实验室);University of Chinese Academy of Sciences(中国科学院大学);University of Amsterdam(阿姆斯特丹大学)

Hongru Song, Yu-an Liu, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Sen Li, Wenjun Peng, Fuyu Lv, Xueqi Cheng

AI总结:

针对商品搜索中稀疏检索的词汇不匹配问题,以及大语言模型应用于稀疏检索的幻觉、训练初始化难问题,提出PROSPER框架,通过字面残差网络和词汇聚焦窗口优化,性能优于稀疏基线、召回率媲美先进稠密检索器,还能提升线上收入。

摘要:

商品搜索是现代电商平台的核心组成部分,每天要处理数十亿条用户查询。在商品搜索系统中,第一阶段检索需要在保证高效线上部署的同时实现高召回率。稀疏检索因具备可解释性与存储高效性,在该场景下极具应用吸引力。然而,稀疏检索方法存在严重的词汇不匹配问题,导致其在商品搜索场景下性能不佳。大语言模型(LLMs)具备语义分析潜力,为缓解词汇不匹配问题、提升检索质量提供了可行方向。\n直接将LLMs应用于商品搜索的稀疏检索面临两大关键挑战:(1)查询和商品标题通常篇幅较短,极易受LLM生成幻觉的影响,比如生成无关的扩展词,或是低估品牌名、型号等关键字面词汇的权重;(2)LLMs的词汇空间庞大,导致训练初始化难以有效开展,很难在这种超高维空间中学习到有意义的稀疏表示。\n为解决上述挑战,我们提出PROSPER——一种利用LLMs作为稀疏检索器的商品搜索框架。PROSPER包含两大模块:(1)字面残差网络:通过残差补偿机制强化被低估的字面词汇,缓解词汇扩展中的幻觉问题;(2)词汇聚焦窗口:通过由粗到细的稀疏化策略,实现有效的训练初始化。\n大量离线与在线实验表明,PROSPER的性能显著优于稀疏基线模型,召回率表现可与先进的稠密检索器相媲美,同时还能带来线上收入的增长。

英文摘要:

Product search is a crucial component of modern e-commerce platforms, with billions of user queries every day. In product search systems, first-stage retrieval should achieve high recall while ensuring efficient online deployment. Sparse retrieval is particularly attractive in this context due to its interpretability and storage efficiency. However, sparse retrieval methods suffer from severe vocabulary mismatch issues, leading to suboptimal performance in product search scenarios. With their potential for semantic analysis, large language models (LLMs) offer a promising avenue for mitigating vocabulary mismatch issues and thereby improving retrieval quality. Directly applying LLMs to sparse retrieval in product search exposes two key challenges:(1)Queries and product titles are typically short and highly susceptible to LLM-induced hallucinations, such as generating irrelevant expansion terms or underweighting critical literal terms like brand names and model numbers;(2)The large vocabulary space of LLMs leads to difficulty in initializing training effectively, making it challenging to learn meaningful sparse representations in such ultra-high-dimensional spaces.To address these challenges, we propose PROSPER, a framework for PROduct search leveraging LLMs as SParsE Retrievers. PROSPER incorporates: (1)A literal residual network that alleviates hallucination in lexical expansion by reinforcing underweighted literal terms through a residual compensation mechanism; and (2)A lexical focusing window that facilitates effective training initialization via a coarse-to-fine sparsification strategy.Extensive offline and online experiments show that PROSPER significantly outperforms sparse baselines and achieves recall performance comparable to advanced dense retrievers, while also achieving revenue increments online.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑