arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.10382cs.LGcs.AI
收藏

基于云协调数据增强的设备学习在推荐系统中的极端模型个性化

On-Device Learning with Cloud-Coordinated Data Augmentation for Extreme Model Personalization in Recommender Systems

Shanghai Jiao Tong University(上海交通大学);University of Texas at Dallas(得克萨斯大学达拉斯分校);Alibaba Group(阿里巴巴集团)

Renjie Gu, Chaoyue Niu, Yikai Yan, Fan Wu, Shaojie Tang, Rongfeng Jia, Chengfei Lyu, Guihai Chen

摘要:

数据异构性是推荐系统的固有属性,使得在云上的全球数据上训练的模型,这是行业的主流,对于每个用户的本地数据分布来说不是最优的。为了处理数据异构性,具有设备上学习的模型个性化是一种潜在的解决方案。然而,使用用户的小规模局部样本进行设备上训练将导致严重的过拟合并破坏模型的泛化能力。在这项工作中,我们提出了一个新的设备-云协作学习框架,称为CoDA,以打破纯粹基于云的学习和设备上学习的困境。CoDA的关键原理是从云的全局池中检索相似的样本,以增强每个用户的本地数据集来训练推荐模型。具体地,在云上进行粗粒度的样本匹配之后,进一步在每个设备上训练个性化的样本分类器进行细粒度的样本过滤,其可以学习本地数据分布和外部数据分布之间的边界。我们还构建了一个端到端的管道,以支持云和每个设备之间的数据流、模型流、计算流和控制流。我们已经在手机淘宝的推荐场景中部署了CoDA。在线A/B测试结果显示,CoDA在没有模型个性化的基于云的学习和没有数据增强的设备上训练方面都有显着的性能提升。在实际设备上的开销测试证明了CoDA中设备上任务的计算,存储和通信效率。

英文摘要:

Data heterogeneity is an intrinsic property of recommender systems, making models trained over the global data on the cloud, which is the mainstream in industry, non-optimal to each individual user's local data distribution. To deal with data heterogeneity, model personalization with on-device learning is a potential solution. However, on-device training using a user's small size of local samples will incur severe overfitting and undermine the model's generalization ability. In this work, we propose a new device-cloud collaborative learning framework, called CoDA, to break the dilemmas of purely cloud-based learning and on-device learning. The key principle of CoDA is to retrieve similar samples from the cloud's global pool to augment each user's local dataset to train the recommendation model. Specifically, after a coarse-grained sample matching on the cloud, a personalized sample classifier is further trained on each device for a fine-grained sample filtering, which can learn the boundary between the local data distribution and the outside data distribution. We also build an end-to-end pipeline to support the flows of data, model, computation, and control between the cloud and each device. We have deployed CoDA in a recommendation scenario of Mobile Taobao. Online A/B testing results show the remarkable performance improvement of CoDA over both cloud-based learning without model personalization and on-device training without data augmentation. Overhead testing on a real device demonstrates the computation, storage, and communication efficiency of the on-device tasks in CoDA.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑