arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.15744cs.LGcs.DCcs.ITmath.IT
收藏

CAFe:成本与年龄感知的联邦学习

CAFe: Cost and Age aware Federated Learning

University of Maryland(马里兰大学);University of Sydney(悉尼大学)

Sahan Liyanaarachchi, Kanchana Thilakarathna, Sennur Ulukus

AI总结:

针对联邦学习中的通信成本与资源浪费问题,提出CAFe方法,利用客户端平均年龄作为收敛指标,优化参数M和T以最小化成本并保证收敛。

摘要:

在许多联邦学习(FL)模型中,一种确保训练过程进展的常见策略是:在参数服务器(PS)广播全局模型后,等待总数为N的客户端中至少M个客户端根据报告截止时间T发回其本地梯度。如果在截止时间内没有足够的客户端报告,则该特定轮次被视为失败轮次,训练轮次从头重新开始。如果有足够的客户端响应,则该轮次被视为成功,所有响应客户端的本地梯度用于更新全局模型。无论哪种情况,未能在截止时间内报告更新的客户端都会浪费其计算资源。更紧的截止时间(较小的T)和等待更多参与客户端(较大的M)会导致大量失败轮次,从而增加通信成本和计算资源浪费。然而,较大的T会导致更长的轮次持续时间,而较小的M可能导致梯度噪声。因此,需要优化参数M和T,以在可接受的收敛速度下最小化通信成本和资源浪费。在这方面,我们表明客户端在PS处的平均年龄明确出现在理论收敛界中,因此可以用作量化全局模型收敛的指标。我们提供了一种分析方案来在此设置中选择参数M和T。

英文摘要:

In many federated learning (FL) models, a common strategy employed to ensure the progress in the training process, is to wait for at least $M$ clients out of the total $N$ clients to send back their local gradients based on a reporting deadline $T$, once the parameter server (PS) has broadcasted the global model. If enough clients do not report back within the deadline, the particular round is considered to be a failed round and the training round is restarted from scratch. If enough clients have responded back, the round is deemed successful and the local gradients of all the clients that responded back are used to update the global model. In either case, the clients that failed to report back an update within the deadline would have wasted their computational resources. Having a tighter deadline (small $T$) and waiting for a larger number of participating clients (large $M$) leads to a large number of failed rounds and therefore greater communication cost and computation resource wastage. However, having a larger $T$ leads to longer round durations whereas smaller $M$ may lead to noisy gradients. Therefore, there is a need to optimize the parameters $M$ and $T$ such that communication cost and the resource wastage is minimized while having an acceptable convergence rate. In this regard, we show that the average age of a client at the PS appears explicitly in the theoretical convergence bound, and therefore, can be used as a metric to quantify the convergence of the global model. We provide an analytical scheme to select the parameters $M$ and $T$ in this setting.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑