arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1802.07876cs.LGcs.IR
收藏

具有快速收敛和高效通信的联邦元学习

Federated Meta-Learning with Fast Convergence and Efficient Communication

Huawei Noah’s Ark Lab(华为诺亚方舟实验室)

Fei Chen, Mi Luo, Zhenhua Dong, Zhenguo Li, Xiuqiang He

摘要:

在移动设备的分布式网络中协作训练机器学习模型的统计和系统挑战一直是联邦学习在现实世界中应用的瓶颈。在这项工作中,我们表明,元学习是一个自然的选择来处理这些问题,并提出了一个联邦元学习框架FedMeta,其中参数化算法(或元学习者)是共享的,而不是在以前的方法中的全局模型。我们对LEAF数据集和真实世界的生产数据集进行了广泛的实证评估,并证明FedMeta实现了所需的通信成本降低了2.82-4.33倍,收敛速度更快,准确率提高了3.23%-14.84%,而Federated Averaging(FedAvg)是联邦学习中领先的优化算法。此外,FedMeta保护用户隐私,因为只有参数化算法在移动设备和中央服务器之间传输,而没有原始数据收集到服务器上。

英文摘要:

Statistical and systematic challenges in collaboratively training machine learning models across distributed networks of mobile devices have been the bottlenecks in the real-world application of federated learning. In this work, we show that meta-learning is a natural choice to handle these issues, and propose a federated meta-learning framework FedMeta, where a parameterized algorithm (or meta-learner) is shared, instead of a global model in previous approaches. We conduct an extensive empirical evaluation on LEAF datasets and a real-world production dataset, and demonstrate that FedMeta achieves a reduction in required communication cost by 2.82-4.33 times with faster convergence, and an increase in accuracy by 3.23%-14.84% as compared to Federated Averaging (FedAvg) which is a leading optimization algorithm in federated learning. Moreover, FedMeta preserves user privacy since only the parameterized algorithm is transmitted between mobile devices and central servers, and no raw data is collected onto the servers.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑