arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.12865cs.LGcs.IRstat.ML
收藏

面向大规模项目推荐的自监督学习

Self-supervised Learning for Large-scale Item Recommendations

Google Inc.(谷歌公司)

Tiansheng Yao, Xinyang Yi, Derek Zhiyuan Cheng, Felix Yu, Ting Chen, Aditya Menon, Lichan Hong, Ed H. Chi, Steve Tjoa, Jieqi Kang, Evan Ettinger

摘要:

大规模推荐模型从海量目录中寻找最相关的项目,在现代搜索和推荐系统中发挥着重要作用。为了对具有大词汇分类特征的输入空间进行建模,典型的推荐模型通过神经网络为来自用户反馈数据的查询和项目学习联合嵌入空间。然而,由于语料库中有数百万到数十亿个条目,用户倾向于为其中非常小的一组提供反馈,从而导致幂律分布。这使得长尾项目的反馈数据非常稀疏。受最近在计算机视觉和自然语言理解中的自监督表示学习研究的成功启发,我们提出了一个用于大规模项目推荐的多任务自监督学习(SSL)框架。该框架旨在通过学习项目特征之间更好的潜在关系来解决标签稀疏问题。具体来说,SSL改进了项目表示学习,并作为额外的正则化来改进泛化。此外,我们提出了一种新的数据增强方法,利用所提出的框架内的特征相关性。我们使用两个真实世界的数据集分别与500 M和1B训练样本评估我们的框架。我们的研究结果表明SSL正则化的有效性,并显示其优越的性能比国家的最先进的正则化技术。我们还已经将所提出的技术应用于网络规模的商业应用到应用推荐系统,并在实时流量的A/B实验中展示了顶级业务指标的显着改进。我们的在线结果也验证了我们的假设,即我们的框架确实在缺乏监督的切片上提高了模型性能。

英文摘要:

Large scale recommender models find most relevant items from huge catalogs, and they play a critical role in modern search and recommendation systems. To model the input space with large-vocab categorical features, a typical recommender model learns a joint embedding space through neural networks for both queries and items from user feedback data. However, with millions to billions of items in the corpus, users tend to provide feedback for a very small set of them, causing a power-law distribution. This makes the feedback data for long-tail items extremely sparse. Inspired by the recent success in self-supervised representation learning research in both computer vision and natural language understanding, we propose a multi-task self-supervised learning (SSL) framework for large-scale item recommendations. The framework is designed to tackle the label sparsity problem by learning better latent relationship of item features. Specifically, SSL improves item representation learning as well as serving as additional regularization to improve generalization. Furthermore, we propose a novel data augmentation method that utilizes feature correlations within the proposed framework. We evaluate our framework using two real-world datasets with 500M and 1B training examples respectively. Our results demonstrate the effectiveness of SSL regularization and show its superior performance over the state-of-the-art regularization techniques. We also have already launched the proposed techniques to a web-scale commercial app-to-app recommendation system, with significant improvements top-tier business metrics demonstrated in A/B experiments on live traffic. Our online results also verify our hypothesis that our framework indeed improves model performance even more on slices that lack supervision.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑