arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.07605cs.LGcs.AI
收藏

UserBERT:通过自我监督对长期和短期用户偏好进行建模

UserBERT: Modeling Long- and Short-Term User Preferences via Self-Supervision

Rakuten Institute of Technology, Rakuten Group, Inc.(乐天集团株式会社乐天技术研究所)

Tianyu Li, Ali Cevahir, Derek Cho, Hao Gong, DuyKhuong Nguyen, Bjorn Stenger

摘要:

电子商务平台每天从数百万独立用户那里生成大量的客户行为数据,例如点击和购买。然而,有效地将这些数据用于行为理解任务是具有挑战性的,因为通常没有足够的标签以监督的方式从所有用户那里学习。本文将BERT模型扩展到电子商务用户数据,以自监督的方式进行预训练表示。通过将序列中的用户操作视为类似于句子中的单词,我们将现有的BERT模型扩展到用户行为数据。此外,我们的模型采用统一的结构,同时从长期和短期用户行为以及用户属性中学习。我们提出了不同类型的用户行为序列的标记化方法,输入表示向量的生成,以及一种新的借口任务,使预训练模型能够从自己的输入中学习,从而消除了对标记训练数据的需要。大量的实验表明,学习的表示在转移到三个不同的现实世界的任务,特别是与特定于任务的建模和多任务表示学习的显着改善

英文摘要:

E-commerce platforms generate vast amounts of customer behavior data, such as clicks and purchases, from millions of unique users every day. However, effectively using this data for behavior understanding tasks is challenging because there are usually not enough labels to learn from all users in a supervised manner. This paper extends the BERT model to e-commerce user data for pre-training representations in a self-supervised manner. By viewing user actions in sequences as analogous to words in sentences, we extend the existing BERT model to user behavior data. Further, our model adopts a unified structure to simultaneously learn from long-term and short-term user behavior, as well as user attributes. We propose methods for the tokenization of different types of user behavior sequences, the generation of input representation vectors, and a novel pretext task to enable the pre-trained model to learn from its own input, eliminating the need for labeled training data. Extensive experiments demonstrate that the learned representations result in significant improvements when transferred to three different real-world tasks, particularly compared to task-specific modeling and multi-task representation learning

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑