Inherit4Rec：推荐模型高效扩展的参数继承
Inherit4Rec: Parameter Inheritance for Efficient Scaling of Recommendation Models
- Kuaishou Technology（快手科技）
- Shenzhen Technology University（深圳技术大学）
- City University of Hong Kong（香港城市大学）
机构由 AI 辅助整理，请以论文原文为准。
中文总结 AI 辅助
针对工业推荐系统模型扩展中的性能与计算瓶颈，提出参数继承框架Inherit4Rec，支持稠密到稠密增长和稠密到稀疏转换，实验验证其优于现有继承基线。
中文摘要 AI 辅助
扩展模型容量已成为克服工业推荐系统性能瓶颈的有效方法。然而，从头开始反复训练更大的稠密模型需要大量的数据和计算时间，而其不断增长的计算量与工业系统的严格服务预算相冲突。参数继承为稠密模型增长和稀疏转换提供了一条有前景的途径，但现有方法主要针对静态语料库设计，在动态演化的推荐数据下可能遭受严重的性能下降。为应对这些挑战，我们提出了Inherit4Rec，一个支持稠密到稠密（D2D）增长和稠密到稀疏（D2S）转换的参数继承框架。Inherit4Rec-D2D结合了混合增长与不对称训练，以在扩展时保持前向函数并维持更新连续性。Inherit4Rec-D2S通过共激活感知分区和负载均衡损失构建SMoE网络，在保留稠密模型能力的同时促进专家激活的均衡。在KuaiRand-1K和工业短视频推荐数据集上的实验表明，两种转换在所有预测目标上均一致优于所评估的继承基线。这些结果证明了Inherit4Rec在工业推荐系统中用于持续容量扩展和计算高效的稀疏转换的有效性。
英文摘要
Scaling model capacity has emerged as an effective approach to overcoming performance bottlenecks in industrial recommender systems. However, repeatedly training larger dense models from scratch demands substantial data and time, while their growing computation conflicts with the strict serving budgets of industrial systems. Parameter inheritance provides a promising route for both dense model growth and sparse conversion, yet existing methods are primarily designed for static corpora and can suffer sharp performance drops under dynamically evolving recommendation data. To address these challenges, we propose Inherit4Rec, a parameter-inheritance framework that supports both Dense-to-Dense (D2D) growth and Dense-to-Sparse (D2S) conversion. Inherit4Rec-D2D combines hybrid growth with asymmetric training to preserve the forward function at expansion and maintain update continuity. Inherit4Rec-D2S constructs SMoE networks through co-activation-aware partitioning and a load-balancing loss, preserving dense-model capabilities while promoting balanced expert activation. Experiments on KuaiRand-1K and an industrial short-video recommendation dataset show that both transformations consistently outperform the evaluated inheritance baselines across all prediction objectives. These results demonstrate the effectiveness of Inherit4Rec for continual capacity expansion and computation-efficient sparse conversion in industrial recommender systems.
