arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.06624cs.LGcs.AI
收藏

设备-云协作学习推荐

Device-Cloud Collaborative Learning for Recommendation

DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Hong Kong Baptist University(香港浸会大学)

Jiangchao Yao, Feng Wang, KunYang Jia, Bo Han, Jingren Zhou, Hongxia Yang

摘要:

随着移动设备上存储和计算能力的快速发展,在设备上部署模型以节省繁重的通信时间并捕获实时特征变得至关重要和流行。虽然已经有相当多的作品探索促进设备上的学习和推理,但它们中的大多数集中在处理响应延迟或隐私保护。在设备和云建模之间的协作建模以及双方共同受益方面做得很少。为了弥合这一差距,我们是第一次尝试研究设备云协作学习(DCCL)框架。具体来说,我们提出了一种新的MetaPatch学习方法,在设备端,有效地实现“成千上万的人与成千上万的模型”给定一个集中的云模型。然后,通过数十亿更新的个性化设备模型,我们提出了一种“模型上的模型”蒸馏算法,即MoMoDistill,以更新集中式云模型。我们在一系列不同设置的数据集上进行了广泛的实验,证明了这种协作在云和设备上的有效性,特别是它在建模长尾用户方面的优势。

英文摘要:

With the rapid development of storage and computing power on mobile devices, it becomes critical and popular to deploy models on devices to save onerous communication latencies and to capture real-time features. While quite a lot of works have explored to facilitate on-device learning and inference, most of them focus on dealing with response delay or privacy protection. Little has been done to model the collaboration between the device and the cloud modeling and benefit both sides jointly. To bridge this gap, we are among the first attempts to study the Device-Cloud Collaborative Learning (DCCL) framework. Specifically, we propose a novel MetaPatch learning approach on the device side to efficiently achieve "thousands of people with thousands of models" given a centralized cloud model. Then, with billions of updated personalized device models, we propose a "model-over-models" distillation algorithm, namely MoMoDistill, to update the centralized cloud model. Our extensive experiments over a range of datasets with different settings demonstrate the effectiveness of such collaboration on both cloud and devices, especially its superiority to model long-tailed users.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑