arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09386cs.IR
收藏

SMES:通过专家稀疏性实现可扩展的多任务推荐

SMES: Towards Scalable Multi-Task Recommendation via Expert Sparsity

Kuaishou Technology Co., Ltd.(快手科技)

Yukun Zhang, Si Dong, Xu Wang, Bo Chen, Qinglin Jia, Shengzhe Wang, Jinlong Jiao, Runhan Li, Jiaqing Liu, Chaoyi Ma, Ruiming Tang, Guorui Zhou, Han Li, Kun Gai

AI总结:

SMES通过稀疏混合专家框架实现可扩展的多任务推荐,解决专家激活爆炸和负载不均问题,提升推荐性能与用户参与度。

摘要:

工业级推荐系统通常依赖多任务学习来估计多样的用户反馈信号并聚合它们以进行排序。模型扩展的最新进展显示了在推荐中的有希望的收益。然而,简单地增加模型容量会带来沉重的在线推理成本,并且对于具有偏斜标签分布的稀疏任务往往会产生递减的回报。这种均匀参数扩展与异构任务容量需求之间的不匹配对可扩展的多任务推荐构成了根本性的挑战。在本文中,我们研究了参数稀疏化作为一种有原则的扩展范式,并识别了在将稀疏混合专家(MoE)应用于多任务推荐时的两个关键障碍:专家激活爆炸导致实例级稀疏性受损以及由任务级路由引起的专家负载不均。为了解决这些挑战,我们提出了SMES,一种可扩展的稀疏MoE框架,具有渐进式的专家路由。SMES将专家激活分解为一个任务共享的专家子集,该子集在跨任务中联合选择,以及任务适应性的私有专家,明确地限制了每个实例的专家执行,同时保持任务特定的容量。此外,SMES引入了一个全局多门负载平衡正则化器,通过调节所有任务的汇总专家使用情况来稳定训练。SMES已部署在快手大规模短视频服务中,支持超过4亿日活跃用户。广泛的在线实验表明,SMES在稳定性方面有所提升,GAUC提升0.29%,用户观看时间提高了0.31%。

英文摘要:

Industrial recommender systems typically rely on multi-task learning to estimate diverse user feedback signals and aggregate them for ranking. Recent advances in model scaling have shown promising gains in recommendation. However, naively increasing model capacity imposes prohibitive online inference costs and often yields diminishing returns for sparse tasks with skewed label distributions. This mismatch between uniform parameter scaling and heterogeneous task capacity demands poses a fundamental challenge for scalable multi-task recommendation. In this work, we investigate parameter sparsification as a principled scaling paradigm and identify two critical obstacles when applying sparse Mixture-of-Experts (MoE) to multi-task recommendation: exploded expert activation that undermines instance-level sparsity and expert load skew caused by independent task-wise routing. To address these challenges, we propose SMES, a scalable sparse MoE framework with progressive expert routing. SMES decomposes expert activation into a task-shared expert subset jointly selected across tasks and task-adaptive private experts, explicitly bounding per-instance expert execution while preserving task-specific capacity. In addition, SMES introduces a global multi-gate load-balancing regularizer that stabilizes training by regulating aggregated expert utilization across all tasks. SMES has been deployed in Kuaishou large-scale short-video services, supporting over 400 million daily active users. Extensive online experiments demonstrate stable improvements, with GAUC gain of 0.29% and a 0.31% uplift in user watch time.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑