arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.09387cs.IR
收藏

混合兴趣提取与异构交互:一种适用于工业推荐系统的可扩展CTR模型

HeMix: Scaling Industrial Ranking Models with Heterogeneous Token Mixing

AMAP, Alibaba Group(高德地图,阿里巴巴集团)

Fangye Wang, Guowei Yang, Xiaojiang Zhou, Song Yang, Pengjie Wang

AI总结:

HeMix是一种适用于工业推荐系统的可扩展CTR模型,通过混合兴趣提取和异构交互结构提升推荐准确性。

摘要:

学习有效的特征交互是现代推荐系统的核心,但在工业环境中仍然具有挑战性,因为存在稀疏的多字段输入和超长的用户行为序列。尽管最近的扩展努力提高了模型容量,但它们往往无法从长期和实时行为序列中构建出具有上下文意识和上下文无关的用户意图。同时,最近的工作也受到效率低和同质交互机制的限制,导致预测性能不佳。为了解决这些限制,我们提出了HeMix,一种可扩展的排序模型,它统一了自适应序列标记化和异构交互结构。具体来说,HeMix引入了一个混合兴趣提取模块,通过动态和固定的查询在全局和实时行为序列上联合建模上下文意识和上下文无关的用户兴趣。在交互方面,我们用HeteroMixer块替代自注意力,使多粒度的跨特征交互成为可能,该交互采用多头标记融合、异构交互和组对齐重建流程。HeMix在HeteroMixer块的驱动下表现出良好的扩展性,通过参数扩展增加模型规模导致推荐准确性稳步提升。在工业级数据集上的实验表明,HeMix能够有效且一致地超越强大的基线。最重要的是,HeMix已在AMAP平台上部署,相较于DLRM,带来了显著的在线收益:GMV增加3.61%,PV_CTR增加2.78%,UV_CVR增加2.12%。

英文摘要:

Scaling up ranking models for industrial recommender systems faces two critical challenges: (C1) existing sequence tokenization fails to jointly capture context-aware and context-invariant user intent from heterogeneous behavior sources, and (C2) prevailing interaction mechanisms are both computationally expensive and semantically homogeneous, limiting prediction quality under strict online latency constraints. We propose \textbf{HeMix}, a scalable ranking model that unifies query-mixed sequence tokenization with heterogeneous feature interaction. To address (C1), HeMix introduces a \textit{Query-Mixed Interest Extraction} module that employs dynamic and fixed queries to simultaneously model context-aware and context-invariant interests from global and real-time behavior sequences. To address (C2), we design the \textit{HeteroMixer} block, comprising Multi-Head Token Fusion, Heterogeneous Mixed-Token Interaction and Group-Aligned Reconstruction, as an efficient alternative to self-attention that enables multi-granularity cross-feature modeling at linear cost. Crucially, HeMix scales smoothly from ${\sim}100$M to ${\sim}1500$M parameters by independently expanding block depth and token dimension, yielding steady accuracy gains without architectural redesign. Experiments on industrial-scale data show that HeMix achieves $+1.64\%$ relative CTR-AUC over the DLRM baseline at the ${\sim}100$M scale while requiring fewer GFLOPs than the strongest competitor. Deployed on the AMAP APP, HeMix yields +0.88\% GMV, +2.74\% PV\_CTR and +0.84\% UV\_CVR over the production baseline in online A/B tests.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑