arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.11226cs.LG
收藏

非独立性在联邦学习中的作用

Demystifying the Effects of Non-Independence in Federated Learning

Institute of Information Systems(信息系统研究所)

Stefan Arnold, Dilara Yesilbas

摘要:

联邦学习(FL)使统计模型能够在用户生成的数据上构建,而不会影响数据安全和用户隐私。出于这个原因,FL非常适合从数据丰富且高度私有化的移动设备上进行设备学习。受移动设备的时间可用性的约束,只有一个子集的设备是可访问的,以参与由训练和聚合组成的迭代协议。在这项研究中,我们采取了一个步骤,更好地了解块循环采样所产生的非独立数据分布的影响。通过对视觉分类进行广泛的实验,我们测量了块循环采样(独立和与非平衡块分布相结合)的影响。具体来说,我们从准确性,公平性和收敛速度的角度来衡量块循环采样引起的变化。实验结果表明,鲁棒性循环两块结构,例如,因为时区的关系。相比之下,依赖于多块结构绘制数据样本会显著降低性能和收敛速度,最高可达26%。此外,我们发现,这种性能退化进一步加剧了不平衡的块分布的点,不能再充分补偿更高的通信和更频繁的同步。

英文摘要:

Federated Learning (FL) enables statistical models to be built on user-generated data without compromising data security and user privacy. For this reason, FL is well suited for on-device learning from mobile devices where data is abundant and highly privatized. Constrained by the temporal availability of mobile devices, only a subset of devices is accessible to participate in the iterative protocol consisting of training and aggregation. In this study, we take a step toward better understanding the effect of non-independent data distributions arising from block-cyclic sampling. By conducting extensive experiments on visual classification, we measure the effects of block-cyclic sampling (both standalone and in combination with non-balanced block distributions). Specifically, we measure the alterations induced by block-cyclic sampling from the perspective of accuracy, fairness, and convergence rate. Experimental results indicate robustness to cycling over a two-block structure, e.g., due to time zones. In contrast, drawing data samples dependently from a multi-block structure significantly degrades the performance and rate of convergence by up to 26%. Moreover, we find that this performance degeneration is further aggravated by unbalanced block distributions to a point that can no longer be adequately compensated by higher communication and more frequent synchronization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑