arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.13183cs.LGcs.CRcs.DC
收藏

通信高效且隐私保护的去中心化元学习

Communication-Efficient and Privacy-Preserving Decentralized Meta-Learning

Hong Kong University of Science and Technology(香港科技大学)

Hansi Yang, James T. Kwok

AI总结:

针对分布式元学习中通信成本高与隐私保护需求,提出引入本地辅助参数和随机扰动的LoDMeta算法,在保证准确率与中心化方法相当的同时,实现通信高效与隐私保护。

摘要:

分布式学习无需将训练数据集中到中心节点,在大数据时代的重要性日益凸显。其中,基于随机游走的去中心化算法具备灵活性,既不需要所有客户端都信任的中心服务器,也不要求所有客户端在每轮迭代中都处于活跃状态。然而,现有分布式学习算法均假设所有学习客户端执行相同任务。本文研究更具挑战性的元学习场景:不同客户端执行不同(但相关)的任务,且各自的训练数据有限。为降低通信成本并实现更优的隐私保护,本文提出LoDMeta(本上去中心化元学习)算法,该算法引入本地辅助优化参数,并对模型参数施加随机扰动。本文从收敛性和隐私分析两方面给出了理论结果。在多个少样本学习数据集上的实验结果表明,LoDMeta的元学习准确率与中心化元学习算法相当,但无需收集各客户端的数据,且能更好地保护每个客户端的数据隐私。

英文摘要:

Distributed learning, which does not require gathering training data in a central location, has become increasingly important in the big-data era. In particular, random-walk-based decentralized algorithms are flexible in that they do not need a central server trusted by all clients and do not require all clients to be active in all iterations. However, existing distributed learning algorithms assume that all learning clients share the same task. In this paper, we consider the more difficult meta-learning setting, in which different clients perform different (but related) tasks with limited training data. To reduce communication cost and allow better privacy protection, we propose LoDMeta (Local Decentralized Meta-learning) with the use of local auxiliary optimization parameters and random perturbations on the model parameter. Theoretical results are provided on both convergence and privacy analysis. Empirical results on a number of few-shot learning data sets demonstrate that LoDMeta has similar meta-learning accuracy as centralized meta-learning algorithms, but does not require gathering data from each client and is able to better protect data privacy for each client.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑