arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2302.02352cs.IR
收藏

TWIN:用于快手CTR预测中终身用户行为建模的两阶段兴趣网络

TWIN: TWo-stage Interest Network for Lifelong User Behavior Modeling in CTR Prediction at Kuaishou

Jianxin Chang, Chenbin Zhang, Zhiyi Fu, Xiaoxue Zang, Lin Guan, Jing Lu, Yiqun Hui, Dewei Leng, Yanan Niu, Yang Song, Kun Gai

AI总结:

针对CTR预测中GSU与ESU相关性指标不一致导致性能下降的问题,提出两阶段兴趣网络TWIN,通过行为特征拆分和缓存策略将一致性注意力机制扩展到长序列行为建模,显著提升了预测准确率。

摘要:

终身用户行为建模,即从数月甚至数年的丰富历史行为中提取用户的隐藏兴趣,在现代CTR预测系统中扮演着核心角色。传统算法大多遵循两个级联阶段:一个简单的通用搜索单元(General Search Unit, GSU),用于对数以万计的长期行为进行快速粗略搜索;以及一个精确搜索单元(Exact Search Unit, ESU),用于对来自GSU的少量最终候选者进行有效的目标注意力(Target Attention, TA)。尽管高效,但现有算法大多存在一个关键局限:GSU和ESU之间目标-行为相关性指标的不一致。因此,它们的GSU通常会遗漏高度相关的行为,却检索出被ESU视为无关的行为。在这种情况下,ESU中的TA无论注意力如何分配,大多会偏离真实的用户兴趣,从而降低整体CTR预测的准确性。为了解决这种不一致性,我们提出了两阶段兴趣网络(TWo-stage Interest Network, TWIN),其中我们的一致性保留GSU(CP-GSU)采用了与ESU中TA相同的目标-行为相关性指标,使两个阶段成为双胞胎。具体而言,为了打破TA的计算瓶颈并将其从ESU扩展到GSU,即从行为长度$10^2$扩展到长度$10^4-10^5$,我们通过行为特征拆分构建了一种新颖的注意力机制。对于行为的视频固有特征,我们通过高效的预计算和缓存策略计算其线性投影。而对于用户-物品交叉特征,我们在注意力分数计算中将每个特征压缩为一维偏置项,以节省计算成本。两个阶段之间的一致性,以及CP-GSU中有效的基于TA的相关性指标,为CTR预测带来了显著的性能提升。

英文摘要:

Life-long user behavior modeling, i.e., extracting a user's hidden interests from rich historical behaviors in months or even years, plays a central role in modern CTR prediction systems. Conventional algorithms mostly follow two cascading stages: a simple General Search Unit (GSU) for fast and coarse search over tens of thousands of long-term behaviors and an Exact Search Unit (ESU) for effective Target Attention (TA) over the small number of finalists from GSU. Although efficient, existing algorithms mostly suffer from a crucial limitation: the \textit{inconsistent} target-behavior relevance metrics between GSU and ESU. As a result, their GSU usually misses highly relevant behaviors but retrieves ones considered irrelevant by ESU. In such case, the TA in ESU, no matter how attention is allocated, mostly deviates from the real user interests and thus degrades the overall CTR prediction accuracy. To address such inconsistency, we propose \textbf{TWo-stage Interest Network (TWIN)}, where our Consistency-Preserved GSU (CP-GSU) adopts the identical target-behavior relevance metric as the TA in ESU, making the two stages twins. Specifically, to break TA's computational bottleneck and extend it from ESU to GSU, or namely from behavior length $10^2$ to length $10^4-10^5$, we build a novel attention mechanism by behavior feature splitting. For the video inherent features of a behavior, we calculate their linear projection by efficient pre-computing \& caching strategies. And for the user-item cross features, we compress each into a one-dimentional bias term in the attention score calculation to save the computational cost. The consistency between two stages, together with the effective TA-based relevance metric in CP-GSU, contributes to significant performance gain in CTR prediction.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑