arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2207.01771cs.LGcs.CRstat.ML
收藏

个性化学习和评估的生成框架:理论、算法和隐私

A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy

University of California, Los Angeles(加州大学洛杉矶分校)

Kaan Ozkara, Antonious M. Girgis, Deepesh Data, Suhas Diggavi

摘要:

联邦学习的一个显著特征是(本地)客户端数据可能具有统计异质性。这种异质性激发了个性化学习的设计,其中通过协作来训练个体(个性化)模型。已经有各种个性化的方法在文献中提出,看似非常不同的形式和方法,从使用一个单一的全球模型的局部正则化和模型插值,使用多个全球模型的个性化聚类等,在这项工作中,我们开始与生成框架,可能会统一几个不同的算法,以及建议新的算法。我们将我们的生成框架应用于个性化估计,并将其与经典的经验贝叶斯方法相结合。我们在这个框架下开发私人个性化的估计。然后,我们使用我们的生成框架的学习,它统一了几个已知的个性化FL算法,并提出了新的,我们提出并研究了一个新的算法AdaPeD的基础上的知识蒸馏,它在数值上优于几个已知的算法。我们还为个性化学习方法开发隐私,保证用户级隐私和组成。我们数值评估的性能以及隐私的估计和学习问题,证明我们所提出的方法的优点。

英文摘要:

A distinguishing characteristic of federated learning is that the (local) client data could have statistical heterogeneity. This heterogeneity has motivated the design of personalized learning, where individual (personalized) models are trained, through collaboration. There have been various personalization methods proposed in literature, with seemingly very different forms and methods ranging from use of a single global model for local regularization and model interpolation, to use of multiple global models for personalized clustering, etc. In this work, we begin with a generative framework that could potentially unify several different algorithms as well as suggest new algorithms. We apply our generative framework to personalized estimation, and connect it to the classical empirical Bayes' methodology. We develop private personalized estimation under this framework. We then use our generative framework for learning, which unifies several known personalized FL algorithms and also suggests new ones; we propose and study a new algorithm AdaPeD based on a Knowledge Distillation, which numerically outperforms several known algorithms. We also develop privacy for personalized learning methods with guarantees for user-level privacy and composition. We numerically evaluate the performance as well as the privacy for both the estimation and learning problems, demonstrating the advantages of our proposed methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑