arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.22101cs.IRcs.LG
收藏

扩展高效小型语言模型的规模化服务与部署以支持语义职位搜索

Scaling Up Efficient Small Language Models Serving and Deployment for Semantic Job Search

Anonymous Authors(匿名作者)
Kayhan Behdin, Qingquan Song, Sriram Vasudevan, Jian Sheng, Xiaojing Ma, Z Zhou, Chuanrui Zhu, Guoyao Li, Chanh Nguyen, Sayan Ghosh, Hejian Sang, Ata Fatahi Baa… 展开作者

Kayhan Behdin, Qingquan Song, Sriram Vasudevan, Jian Sheng, Xiaojing Ma, Z Zhou, Chuanrui Zhu, Guoyao Li, Chanh Nguyen, Sayan Ghosh, Hejian Sang, Ata Fatahi Baarzi, Sundara Raman Ramachandran, Xiaoqing Wang, Qing Lan, Vinay Y S, Qi Guo, Caleb Johnson, Zhipeng Wang, Fedor Borisyuk

AI总结:

本文分享了在LinkedIn为语义职位搜索开发小型语言模型时,通过剪枝和上下文压缩技术分别将模型大小减少40%和输入长度减少10倍,并在GPU上优化服务,最终实现吞吐量提升10倍的经验。

摘要:

大型语言模型(LLMs)在应用于相关性排序和语义搜索等预测任务时已展现出卓越的质量。然而,对于具有严格延迟和吞吐量要求的行业应用而言,部署此类LLM的成本仍然高得令人望而却步。在这项工作中,我们分享了在LinkedIn为语义搜索应用开发纯文本解码器专用小型语言模型(SLM)过程中获得的经验教训和效率见解。具体而言,我们讨论了诸如剪枝等模型压缩技术,这些技术使我们能够将模型大小减少高达40%,同时保持准确性。此外,我们提出了上下文压缩技术,这些技术使我们能够将输入上下文长度减少高达10倍,且准确率损失极小。最后,我们分享了在GPU上大规模优化服务基础设施以部署此类系统(每秒处理数百万请求)的实践经验。综合来看,这些方法使我们在实际部署中将系统吞吐量提高了10倍,同时满足了我们的质量要求。

英文摘要:

Large Language Models (LLMs) have demonstrated impressive quality when applied to predictive tasks such as relevance ranking and semantic search. However, deployment of such LLMs remains prohibitively expensive for industry applications with strict latency and throughput requirements. In this work, we present lessons and efficiency insights from developing a purely text-based decoder-only Small Language Model (SLM) for a semantic search application at LinkedIn. Particularly, we discuss model compression techniques such as pruning that allow us to reduce the model size by up to $40\%$ while maintaining the accuracy. Additionally, we present context compression techniques that allow us to reduce the input context length by up to $10$x with minimal loss of accuracy. Finally, we present practical lessons from optimizing the serving infrastructure for deploying such a system on GPUs at scale, serving millions of requests per second. Taken together, this allows us to increase our system's throughput by $10$x in a real-world deployment, while meeting our quality bar.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑