arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.14110cs.IR
收藏

MixFormer: 在工业推荐系统中协同扩展稠密与序列

MixFormer: Co-Scaling Up Dense and Sequence in Industrial Recommenders

ByteDance Inc.(字节跳动)

Xu Huang, Hao Zhang, Zhifang Fan, Yunwen Huang, Zhuoxing Wei, Zheng Chai, Jinan Ni, Yuchao Zheng, Qiwei Chen

AI总结:

提出MixFormer统一Transformer架构,联合建模序列行为与特征交互,通过统一参数化实现稠密容量与序列长度的协同扩展,并引入用户-物品解耦策略提升工业实用性。

摘要:

随着工业推荐系统进入扩展驱动时代,Transformer架构因其能够将模型向更大容量和更长序列扩展而日益受到青睐。然而,现有的基于Transformer的推荐模型在结构上仍然碎片化,序列建模和特征交互作为独立模块实现,具有独立的参数化。这种设计引入了一个基本的协同扩展挑战,因为在有限的计算预算下,模型容量必须在稠密特征交互和序列建模之间进行次优分配。在这项工作中,我们提出了MixFormer,一种专为推荐系统设计的统一Transformer风格架构,它在单个主干中联合建模序列行为和特征交互。通过统一参数化,MixFormer实现了稠密容量和序列长度的有效协同扩展,缓解了解耦设计中观察到的权衡。此外,集成架构促进了序列和非序列表示之间的深度交互,使得高阶特征语义能够直接指导序列聚合,从而增强整体表达能力。为了确保工业实用性,我们进一步引入了一种用户-物品解耦策略进行效率优化,显著减少了冗余计算和推理延迟。在大规模工业数据集上的广泛实验表明,MixFormer始终表现出优越的准确性和效率。此外,在两个生产推荐系统——抖音和抖音极速版上的大规模在线A/B测试显示,用户参与度指标(包括活跃天数和应用内使用时长)持续改善。

英文摘要:

As industrial recommender systems enter a scaling-driven regime, Transformer architectures have become increasingly attractive for scaling models towards larger capacity and longer sequence. However, existing Transformer-based recommendation models remain structurally fragmented, where sequence modeling and feature interaction are implemented as separate modules with independent parameterization. Such designs introduce a fundamental co-scaling challenge, as model capacity must be suboptimally allocated between dense feature interaction and sequence modeling under a limited computational budget. In this work, we propose MixFormer, a unified Transformer-style architecture tailored for recommender systems, which jointly models sequential behaviors and feature interactions within a single backbone. Through a unified parameterization, MixFormer enables effective co-scaling across both dense capacity and sequence length, mitigating the trade-off observed in decoupled designs. Moreover, the integrated architecture facilitates deep interaction between sequential and non-sequential representations, allowing high-order feature semantics to directly inform sequence aggregation and enhancing overall expressiveness. To ensure industrial practicality, we further introduce a user-item decoupling strategy for efficiency optimizations that significantly reduce redundant computation and inference latency. Extensive experiments on large-scale industrial datasets demonstrate that MixFormer consistently exhibits superior accuracy and efficiency. Furthermore, large-scale online A/B tests on two production recommender systems, Douyin and Douyin Lite, show consistent improvements in user engagement metrics, including active days and in-app usage duration.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑