arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.04641cs.LGcs.AI
收藏

可能近似正确的联邦学习

Probably Approximately Correct Federated Learning

Hong Kong University of Science and Technology(香港科技大学);Webank(微众银行)

Xiaojin Zhang, Anbu Huang, Lixin Fan, Kai Chen, Qiang Yang

AI总结:

提出FedPAC框架,利用PAC学习以样本复杂度量化联邦学习中的隐私、效用和效率目标,将多目标优化转化为单目标优化,从而高效求解最优权衡方案。

摘要:

联邦学习(FL)是一种新的分布式学习范式,以隐私、效用和效率为其主要支柱。现有研究表明,同时实现无穷小的隐私泄漏、效用损失和低效率是不太可能的。因此,如何找到最优的权衡解决方案是设计FL算法时的关键考虑因素。一种常见的方法是将权衡问题转化为多目标优化问题,即目标是在将隐私泄漏限制在不超过预定值的同时,最小化效用损失和效率降低。然而,现有的多目标优化框架非常耗时,并且不能保证Pareto前沿的存在,这促使我们寻找一种将多目标问题转化为单目标问题的解决方案,因为它更高效且更容易求解。为此,我们提出了FedPAC,这是一个利用PAC学习以样本复杂度量化多个目标的统一框架,这种量化允许我们将多个目标的解空间约束到一个共享维度,从而可以通过单目标优化算法来求解。具体而言,我们从PAC学习的角度提供了如何量化效用损失、隐私泄漏、隐私-效用-效率权衡以及攻击者成本的结果和详细分析。

英文摘要:

Federated learning (FL) is a new distributed learning paradigm, with privacy, utility, and efficiency as its primary pillars. Existing research indicates that it is unlikely to simultaneously attain infinitesimal privacy leakage, utility loss, and efficiency. Therefore, how to find an optimal trade-off solution is the key consideration when designing the FL algorithm. One common way is to cast the trade-off problem as a multi-objective optimization problem, i.e., the goal is to minimize the utility loss and efficiency reduction while constraining the privacy leakage not exceeding a predefined value. However, existing multi-objective optimization frameworks are very time-consuming, and do not guarantee the existence of the Pareto frontier, this motivates us to seek a solution to transform the multi-objective problem into a single-objective problem because it is more efficient and easier to be solved. To this end, we propose FedPAC, a unified framework that leverages PAC learning to quantify multiple objectives in terms of sample complexity, such quantification allows us to constrain the solution space of multiple objectives to a shared dimension, so that it can be solved with the help of a single-objective optimization algorithm. Specifically, we provide the results and detailed analyses of how to quantify the utility loss, privacy leakage, privacy-utility-efficiency trade-off, as well as the cost of the attacker from the PAC learning perspective.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑