FedFIbOS：基于Fisher重要性的异构联邦学习最优子建模
FedFIbOS: Fisher Importance based Optimal Submodelling for Heterogeneous Federated Learning
- University of Otago（奥塔哥大学）
- University of New South Wales（新南威尔士大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
针对异构联邦学习子模型参数选择缺乏理论依据的问题，提出基于Fisher重要性的最优子建模方法FedFIbOS，通过最小化掩蔽误差并高效估计Fisher分数，在多个数据集上比现有方法准确率提升约10%。
AI中文摘要：
异构联邦学习要求具有不同计算能力的客户端协作训练一个全局模型，其中每个客户端训练一个受容量约束的子模型。现有方法使用启发式重要性度量（最显著的是参数幅度）来选择子模型参数，但缺乏这些度量为何支持收敛的理论依据。我们发现了一个根本性差距：现有的参数选择标准在收敛框架中缺乏理论基础，且部分客户端参与在Fisher分数中引入了额外的估计效应。我们提出了FedFIbOS：用于异构联邦学习的基于Fisher重要性的最优子建模，使用Fisher信息作为从最小化子模型掩蔽误差推导出的原则性准则。我们从理论上将子模型选择表述为Fisher加权二次掩蔽代理，并表明FedFIbOS实现的原始Fisher top-k规则在Fisher主导排序条件下解决了该代理。所得到的方法保留了底层掩蔽联邦优化界限的收敛结构。Fisher分数通过平方梯度从经验对角Fisher信息中高效估计，从而实现稳定且自适应的参数选择，无需额外的优化开销。在CIFAR-10、CIFAR-100和AGNews上，在病理性和Dirichlet非IID设置下的实验表明，FedFIbOS比现有最先进方法实现了约10%的更高准确率，且在更强的异构性下改进更为显著。
英文摘要：
Heterogeneous federated learning requires clients with diverse computational capacities to collaboratively train a global model, where each client trains a capacity-constrained submodel. Existing methods select submodel parameters using heuristic importance measures---most prominently parameter magnitude---without theoretical justification for why these measures support convergence. We identify a fundamental gap: existing parameter selection criteria lack theoretical grounding in the convergence framework, partial client participation introduces additional estimation effects in the Fisher scores. We propose \textbf{FedFIbOS}: Fisher Importance-based Optimal Submodelling for heterogeneous federated learning, using Fisher Information in a principled criterion derived from minimizing submodel masking error. %We formally establish when magnitude selection is equivalent to Fisher selection fail under non-IID heterogeneous federated learning. We theoretically formulate submodel selection through a Fisher-weighted quadratic masking surrogate and show that the raw Fisher top-$k$ rule implemented by FedFIbOS solves this surrogate under a Fisher-dominant ranking condition. The resulting method retains the convergence structure of the underlying masked federated optimization bound. Fisher scores are efficiently estimated from empirical diagonal Fisher information using squared gradients, enabling stable and adaptive parameter selection without additional optimization overhead. Experiments on CIFAR-10, CIFAR-100, and AGNews under pathological and Dirichlet non-IID settings show FedFIbOS achieves ${\approx}10\%$ higher accuracy than the state of the art, with improvements becoming more pronounced under stronger heterogeneity.
