发表机构
University of California, Santa Barbara（加州大学圣塔芭芭拉分校）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本文提出一种结合可学习软Top-K、逐项阈值化和FLOPs正则化的自适应稀疏性优化方案，在MS MARCO和BEIR数据集上显著缩短查询和文档长度，降低检索延迟与存储成本，同时保持高相关性。
AI 中文摘要
近期神经稀疏检索的研究通过利用大型语言模型（LLMs）进行语义术语扩展，展现了强大的相关性。然而，与先前稀疏化技术配对的学习模型，由于大型LLM词汇表，仍会产生过长的文档和查询向量，对检索时间和空间效率构成严峻挑战。本文提出了一种通过自适应策略协同优化模型稀疏性的方案，包括可学习软Top-K、逐项阈值化和FLOPs正则化，以增加查询和文档向量的稀疏性。在MS MARCO和BEIR数据集上使用Lion-SP模型的实验结果表明，所提方案能显著减少平均查询和文档长度，从而超越基线。我们的方案能够实现更短的检索延迟和更低的存储成本，同时保持极具竞争力的相关性。
英文摘要
Recent work on neural sparse retrieval has demonstrated strong relevance by leveraging Large Language Models (LLMs) for semantic term expansion. However, learned models paired with previous sparsification techniques still yield overly long document and query vectors partly due to a large LLM vocabulary, imposing a serious challenge to retrieval time and space efficiency. This paper proposes a scheme for optimizing model sparsity through a synergy of adaptive strategies, including learnable soft top-K, per-term thresholding, and FLOPs regularization to increase the sparsity of query and document vectors. Experimental results with Lion-SP model on the MS MARCO and BEIR datasets demonstrate that the proposed scheme can outperform the baselines by significantly reducing the average query and document lengths. Our scheme can achieve much shorter retrieval latency and lower storage cost while maintaining highly competitive relevance.
CommentsAccepted at SIGIR 2026
Journal refProc. SIGIR '26 (2026) 2072-2083
