arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2610.28565cs.LGmath.DS
收藏

聚类联邦学习中的崩溃与恢复动力学

The Dynamics of Collapse and Recovery in Clustered Federated Learning

University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)

Priyanka Sinha, Snigdhansu Chatterjee

AI总结:

该研究针对聚类联邦学习的崩溃问题,将其建模为未知动力学系统的控制问题,提出基于阻塞EXP3的自主控制器选择方法,可从不利起始条件恢复可用专家,解决了标准CFL易崩溃的问题。

摘要:

聚类联邦学习(CFL)可能会出现聚类崩溃问题。具体而言,在不利的初始化条件下,早期的客户端分配会将异构组集中到一个聚类模型上,该模型随后会被训练为折中模型,而另一个模型则无法获得更新。由于当前模型决定客户端路由,而路由又决定后续模型更新,因此我们将聚类联邦学习建模为对未知动力学系统的控制问题,其中聚类模型为演化状态,客户端路由模式为控制输入,路由干预为依赖状态的控制器。在两组线性模型中,我们确定了一个初始化区域,其中贪心路由会使崩溃自我强化。随后我们推导了有限时间误差动力学,表明路由干预可通过反复训练被“饥饿”的模型(主要使用其兼容组的更新)来逆转崩溃。受此条件启发,每个客户端使用阻塞EXP3在贪心路由和将更新重定向到未使用模型的干预之间进行选择。实验表明,通过更简单的预测器和更快的专家形成,自主控制器选择可从不利起始条件中恢复可用专家,而标准CFL基线会陷入该不利起始条件。在转换后的FEMNIST数据集中,具有持续承诺的同步输入配置可在自主控制器配置常无法做到的场景中恢复可用专家。用于复现实验和图表的代码与脚本可在该https链接获取。

英文摘要:

Clustered federated learning (CFL) can suffer from cluster collapse. In particular, under unfavorable initialization, early client assignments concentrate heterogeneous groups on one cluster model, which is then trained toward a pooled compromise, while another model receives no updates. Since the current models determine client routing and routing determines subsequent model updates, we formulate clustered federated learning as control of an unknown dynamical system, with cluster models as the evolving state, client routing patterns as the control input, and routing interventions as state dependent controllers. In a two group linear model, we identify an initialization region in which greedy routing makes collapse self reinforcing. We then derive finite horizon error dynamics showing how routing interventions can reverse collapse by repeatedly training the starved model, primarily with updates from its compatible group. Motivated by this condition, each client uses blocked EXP3 to choose between greedy routing and interventions that redirect updates toward unused models. Experiments show that with simpler predictors and faster specialist formation, autonomous controller selection recovers usable specialists from unfavorable starts that trap standard CFL baselines. In transformed FEMNIST, the synchronized-entry configuration with persistent commitment recovers usable specialists in settings where the autonomous controller configuration often does not. Code and scripts for reproducing the experiments and figures are available at https://github.com/prsinha1/FedRecover.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑