MaskNet:通过实例引导掩码将智能乘法引入CTR排名模型
MaskNet: Introducing Feature-Wise Multiplication to CTR Ranking Models by Instance-Guided Mask
摘要:
点击率(CTR)估计已成为许多实际应用中最基本的任务之一,而有效地捕捉复杂的高阶特征对排序模型至关重要。浅层前馈网络广泛用于许多最先进的DNN模型,如FNN,DeepFM和xDeepFM,以隐式地捕获高阶特征交互。然而,一些研究已经证明,成瘾的特征交互,特别是前馈神经网络,是低效的捕获共同的特征交互。为了解决这个问题,我们通过提出实例引导的掩模将特定的乘法运算引入DNN排名系统,该掩模在输入实例引导的特征嵌入层和前馈层上执行逐元素积。本文还提出了MaskBlock算法,将DNN模型中的前馈层转化为加法和乘法特征交互的混合。MaskBlock结合了层归一化、实例引导掩码和前馈层,它是用于在各种配置下设计新的排名模型的基本构建块。本文将MaskBlock构成的模型称为MaskNet,并提出了两个新的MaskNet模型,以证明MaskBlock作为构成高性能排名系统的基本构件的有效性。在三个真实数据集上的实验结果表明,我们提出的MaskNet模型的性能明显优于DeepFM和xDeepFM等最先进的模型,这意味着MaskBlock是组成新的高性能排名系统的有效基本构建单元。
英文摘要:
Click-Through Rate(CTR) estimation has become one of the most fundamental tasks in many real-world applications and it's important for ranking models to effectively capture complex high-order features. Shallow feed-forward network is widely used in many state-of-the-art DNN models such as FNN, DeepFM and xDeepFM to implicitly capture high-order feature interactions. However, some research has proved that addictive feature interaction, particular feed-forward neural networks, is inefficient in capturing common feature interaction. To resolve this problem, we introduce specific multiplicative operation into DNN ranking system by proposing instance-guided mask which performs element-wise product both on the feature embedding and feed-forward layers guided by input instance. We also turn the feed-forward layer in DNN model into a mixture of addictive and multiplicative feature interactions by proposing MaskBlock in this paper. MaskBlock combines the layer normalization, instance-guided mask, and feed-forward layer and it is a basic building block to be used to design new ranking model under various configurations. The model consisting of MaskBlock is called MaskNet in this paper and two new MaskNet models are proposed to show the effectiveness of MaskBlock as basic building block for composing high performance ranking systems. The experiment results on three real-world datasets demonstrate that our proposed MaskNet models outperform state-of-the-art models such as DeepFM and xDeepFM significantly, which implies MaskBlock is an effective basic building unit for composing new high performance ranking systems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。