arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.03578cs.IR
收藏

顺序推荐中边信息融合的无创自注意算法

Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation

Chang Liu, Xiaoguang Li, Guohao Cai, Zhenhua Dong, Hong Zhu, Lifeng Shang

摘要:

顺序推荐系统的目标是从用户的历史行为中建模用户不断变化的兴趣,从而定制时间相关的推荐。与传统模型相比,CNN和RNN等深度学习方法在推荐任务方面取得了显着进步。最近,BERT框架也成为一种很有前途的方法,得益于其在处理序列数据时的自注意力机制。然而,原始BERT框架的一个限制是它只考虑自然语言标记的一个输入源。在BERT框架下如何利用各种类型的信息仍然是一个悬而未决的问题。尽管如此,利用其他辅助信息(例如项目类别或标签)来进行更全面的提示和更好的推荐在直觉上是有吸引力的。在我们的试点实验中,我们发现直接将副信息类型融合到项目嵌入中的天真方法通常带来很少甚至负面的影响。因此,在本文中,我们在BERT框架下,提出了一种有效利用边信息的非虚自注意力机制(NOVA). NOVA利用边信息来产生更好的注意分配,而不是直接改变项目嵌入,这可能会导致信息压倒性。我们在公共和商业数据集上验证了NOVA-BERT模型,我们的方法可以稳定地优于最先进的模型,计算开销可以忽略不计。

英文摘要:

Sequential recommender systems aim to model users' evolving interests from their historical behaviors, and hence make customized time-relevant recommendations. Compared with traditional models, deep learning approaches such as CNN and RNN have achieved remarkable advancements in recommendation tasks. Recently, the BERT framework also emerges as a promising method, benefited from its self-attention mechanism in processing sequential data. However, one limitation of the original BERT framework is that it only considers one input source of the natural language tokens. It is still an open question to leverage various types of information under the BERT framework. Nonetheless, it is intuitively appealing to utilize other side information, such as item category or tag, for more comprehensive depictions and better recommendations. In our pilot experiments, we found naive approaches, which directly fuse types of side information into the item embeddings, usually bring very little or even negative effects. Therefore, in this paper, we propose the NOninVasive self-attention mechanism (NOVA) to leverage side information effectively under the BERT framework. NOVA makes use of side information to generate better attention distribution, rather than directly altering the item embedding, which may cause information overwhelming. We validate the NOVA-BERT model on both public and commercial datasets, and our method can stably outperform the state-of-the-art models with negligible computational overheads.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑