异构环境中用于有效联邦学习的分层知识结构化
Hierarchical Knowledge Structuring for Effective Federated Learning in Heterogeneous Environments
AI总结：
针对联邦学习中非独立同分布数据导致直接聚合logit产生有偏知识的问题，提出分层知识结构化（HKS）框架，通过多粒度码本和无监督聚类实现全局与本地知识的有效平衡，提升了模型鲁棒性和知识共享效果。
摘要：
联邦学习能够在保持个体数据隐私的同时，实现跨分布式实体的协作模型训练。联邦学习中的一个关键挑战是平衡本地客户端模型的个性化与全局模型的泛化能力。最近的研究利用基于logit的知识聚合和蒸馏来克服这些问题。然而，由于不同客户端间数据的非独立同分布（non-IID）特性以及客户端数据分布的不平衡，直接聚合logit通常会产生有偏的知识，这些知识无法适用于各个客户端，并阻碍了本地训练的收敛。为了解决这个问题，我们提出了一种分层知识结构化（HKS）框架，该框架将样本logit构建为多粒度码本，以表示从个性化的每样本洞察到全局化的每类知识的logit。利用无监督的自底向上聚类方法，使全局服务器能够向本地客户端提供多粒度响应。这些响应允许本地训练将监督学习目标与全局泛化约束相结合，从而产生更鲁棒的表示，并在后续训练轮次中改善知识共享。所提出框架的有效性在各种基准和模型架构上得到了验证。
英文摘要：
Federated learning enables collaborative model training across distributed entities while maintaining individual data privacy. A key challenge in federated learning is balancing the personalization of models for local clients with generalization for the global model. Recent efforts leverage logit-based knowledge aggregation and distillation to overcome these issues. However, due to the non-IID nature of data across diverse clients and the imbalance in the client's data distribution, directly aggregating the logits often produces biased knowledge that fails to apply to individual clients and obstructs the convergence of local training. To solve this issue, we propose a Hierarchical Knowledge Structuring (HKS) framework that formulates sample logits into a multi-granularity codebook to represent logits from personalized per-sample insights to globalized per-class knowledge. The unsupervised bottom-up clustering method is leveraged to enable the global server to provide multi-granularity responses to local clients. These responses allow local training to integrate supervised learning objectives with global generalization constraints, which results in more robust representations and improved knowledge sharing in subsequent training rounds. The proposed framework's effectiveness is validated across various benchmarks and model architectures.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
