发表机构
Institute of Science Tokyo（东京科学大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出Tram-FL机制，通过顺序循环单个模型并优化调度与量化动量，在去中心化联邦学习中同时降低通信和计算成本，并在非IID数据下保持高精度。
AI 中文摘要
传统的去中心化联邦学习（DFL）通常以客户端为中心，每个客户端维护一个模型副本，独立执行更新，并进行模型交换与集成。虽然充分利用计算资源可以缩短训练时间，但也可能导致显著的计算和通信浪费。在非独立同分布（non-IID）数据下尤为明显，因为要达到高模型精度需要额外资源。本研究将焦点转向模型本身，旨在以最小的计算和通信成本实现DFL。为此，我们提出了Tram-FL（去中心化联邦学习的旅行模型训练机制），该机制旨在高效解决这些挑战。它通过在节点间循环单个模型来顺序训练该模型。我们解决了基于模型循环的训练中的调度问题，具体确定哪些节点应更新模型以及执行的更新次数。这通过考虑模型的循环路径和更新迭代分配来实现，为此我们提出了简单而有效的方法。此外，借助量化动量，Tram-FL在控制每次传输通信负载的同时，以较少的模型循环实现高精度。实验结果表明，所提出的算法即使在non-IID数据下，也能以降低的通信和计算成本收敛到全局模型。
英文摘要
Conventional decentralized federated learning (DFL) often focuses on clients, with each client maintaining a model copy, performing updates individually, and undertaking model exchange and integration. While fully leveraging computational resources can shorten training times, it can also lead to significant computational and communication waste. This is especially pronounced with non-independent and identically distributed (non-IID) data, where achieving high model accuracy demands extra resources. This research shifts focus to the model itself, aiming to realize DFL with minimal computation and communication costs. To this end, we propose Tram-FL (Traveling Model Training Mechanism for Decentralized Federated Learning), a mechanism designed to efficiently address these challenges. It sequentially trains a single model by circulating it among nodes. We address the training scheduling problem in model circulation-based training, specifically determining which nodes should update the model and the number of updates to perform. This is approached by considering the model's circulation route and update iteration allocation, for which we propose simple yet effective methods. Additionally, with quantized momentum, Tram-FL achieves high accuracy with fewer model circulations while controlling communication load per transmission. Experimental results show that the proposed algorithm, even with non-IID data, converges to a global model with reduced communication and computation.
Comments12 pages, 7 figures, 5 tables. This work has been submitted to the IEEE for possible publication
