融合异构神经网络中的神经元干扰研究
Investigating Neuron Disturbing in Fusing Heterogeneous Neural Networks
AI总结：
本文从贝叶斯视角揭示异构局部模型融合中的神经元干扰现象，并提出自适应选择局部模型进行预测的AMS方法，实验证明其在数据异质性下比一般融合与集成方法更鲁棒。
摘要：
在单次通信轮次中，将分布在各自客户端上训练的深度学习模型融合为一个全局模型，是联邦学习的一种直接实现方式。尽管当前的模型融合方法在实验上被证明能够有效融合架构几乎相同的神经网络，但很少从理论上加以分析。本文揭示了神经元干扰现象，即来自异构局部模型的神经元会相互干扰。我们结合客户端之间的数据异质性以及神经网络的特性，从贝叶斯视角给出了详细解释。此外，为了验证我们的发现，我们提出了一种排除神经元干扰的实验方法，该方法通过自适应选择一个局部模型来根据输入执行预测，称为AMS。实验表明，在数据异质性条件下，AMS比一般的模型融合方法和集成方法更具鲁棒性。这意味着在模型融合中考虑神经元干扰的必要性。此外，AMS作为一种实验算法可用于融合具有不同架构的模型，我们还列出了AMS未来工作的若干可能扩展方向。
英文摘要：
Fusing deep learning models trained on separately located clients into a global model in a one-shot communication round is a straightforward implementation of Federated Learning. Although current model fusion methods are shown experimentally valid in fusing neural networks with almost identical architectures, they are rarely theoretically analyzed. In this paper, we reveal the phenomenon of neuron disturbing, where neurons from heterogeneous local models interfere with each other mutually. We give detailed explanations from a Bayesian viewpoint combining the data heterogeneity among clients and properties of neural networks. Furthermore, to validate our findings, we propose an experimental method that excludes neuron disturbing and fuses neural networks via adaptively selecting a local model, called AMS, to execute the prediction according to the input. The experiments demonstrate that AMS is more robust in data heterogeneity than general model fusion and ensemble methods. This implies the necessity of considering neural disturbing in model fusion. Besides, AMS is available for fusing models with varying architectures as an experimental algorithm, and we also list several possible extensions of AMS for future work.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
