MSN：一种基于记忆的稀疏激活缩放框架用于大规模工业推荐
MSN: A Memory-based Sparse Activation Scaling Framework for Large-scale Industrial Recommendation
AI总结：
MSN通过基于记忆的稀疏激活缩放框架实现大规模工业推荐中的高效个性化与性能提升。
摘要：
深度学习推荐模型的扩展是提高模型表达能力的有效方法。现有方法往往带来显著的计算开销，使其难以在严格的延迟约束下部署于大规模工业系统中。最近的稀疏激活缩放方法，如稀疏混合专家（Sparse Mixture-of-Experts），通过只激活参数子集来减少计算，但仍然由于专家规模大且数量少而面临高内存访问成本和有限的个性化能力。为了解决这些挑战，我们提出了MSN，一种用于推荐模型的基于记忆的稀疏激活缩放框架。MSN动态从大量参数化的记忆中检索个性化表示，并通过记忆门控机制将其整合到下游特征交互模块中，从而实现低计算开销下的细粒度个性化。为了在扩大内存容量的同时保持计算和内存访问成本的控制，MSN采用产品密钥记忆（Product-Key Memory，PKM）机制，将内存检索复杂度从线性时间降至亚线性时间。此外，引入归一化和过参数化技术以保持内存利用率的平衡并防止内存检索崩溃。我们进一步设计了定制化的稀疏收集运算符，并采用AirTopK运算符以提高工业环境中的训练和推理效率。广泛的实验表明，MSN在保持高效率的同时，始终提高了推荐性能。此外，MSN已在抖音搜索排名系统中成功部署，在离线评估指标和大规模在线A/B测试中均显著优于已部署的最先进模型。
英文摘要：
Scaling deep learning recommendation models is an effective way to improve model expressiveness. Existing approaches often incur substantial computational overhead, making them difficult to deploy in large-scale industrial systems under strict latency constraints. Recent sparse activation scaling methods, such as Sparse Mixture-of-Experts, reduce computation by activating only a subset of parameters, but still suffer from high memory access costs and limited personalization capacity due to the large size and small number of experts. To address these challenges, we propose MSN, a memory-based sparse activation scaling framework for recommendation models. MSN dynamically retrieves personalized representations from a large parameterized memory and integrates them into downstream feature interaction modules via a memory gating mechanism, enabling fine-grained personalization with low computational overhead. To enable further expansion of the memory capacity while keeping both computational and memory access costs under control, MSN adopts a Product-Key Memory (PKM) mechanism, which factorizes the memory retrieval complexity from linear time to sub-linear complexity. In addition, normalization and over-parameterization techniques are introduced to maintain balanced memory utilization and prevent memory retrieval collapse. We further design customized Sparse-Gather operator and adopt the AirTopK operator to improve training and inference efficiency in industrial settings. Extensive experiments demonstrate that MSN consistently improves recommendation performance while maintaining high efficiency. Moreover, MSN has been successfully deployed in the Douyin Search Ranking System, achieving significant gains over deployed state-of-the-art models in both offline evaluation metrics and large-scale online A/B test.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
