用于构建高效个性化模型的全局联邦学习策略
Global Federated Learning Strategies for Building Efficient Personalized Models
- Kim Jaechul Graduate School of AI（金宰哲人工智能研究生院）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
该研究针对联邦学习中全局与个性化性能同时恶化的问题，提出缓解特征向量差异、结合特征蒸馏及调整全局初始化策略的方法，实现了全局知识保留与个性化性能的平衡。
AI中文摘要：
联邦学习（FL）是一种可在分布式用户数据上训练模型同时保障数据隐私的实用框架；但由于每个用户的数据分布存在异质性，常出现全局性能与个性化性能同时恶化的问题。本论文提出构建高效个性化模型的方法，通过识别全局训练阶段有效的策略，以及展示如何在局部适配时保留全局知识同时保障用户特定性能。首先，研究表明随着数据异质性增加，特征向量崩溃是比分类器权重更根本的瓶颈，提出直接缓解局部与全局模型间表示幅度差异的方法。其次，分析发现强化局部对齐的训练方法会导致全局知识（如局部未观测到的类别）遗忘，提出结合基于全局模型特征向量的特征蒸馏，以同时实现局部对齐与全局知识保留的方法。第三，在存在偏好异质性的联邦个性化奖励模型学习中，实证验证“增加全局模型数量可获得更好初始化”的传统观点，并表明当允许充分的局部微调时，单一全局初始化反而能提供更强的个性化性能。本研究重新定义了数据与偏好异质性下全局初始化的作用，提供同时满足全局知识保留与个性化需求的实用训练策略。
英文摘要：
Federated learning (FL) is a practical framework that can train models on distributed user data while guaranteeing data privacy; however, due to heterogeneity in which each user has a different data distribution, problems frequently arise where both global and personalization performance deteriorate simultaneously. This dissertation presents methodologies for building efficient personalized models by identifying which strategies are effective in the global training stage and by showing how to preserve global knowledge while securing user-specific performance during local adaptation. First, we show that as data heterogeneity increases, the collapse of feature vectors is a more fundamental bottleneck than classifier weights, and propose a method that directly mitigates the discrepancy in representation magnitude between local and global models. Second, we analyze that a training approach that strengthens local alignment can induce forgetting of global knowledge (e.g., categories not observed locally), and propose a method that achieves both local alignment and global knowledge preservation by combining feature distillation based on the global model's feature vectors. Third, in federated personalized reward model learning with preference heterogeneity, we empirically verify the conventional belief that "increasing the number of global models yields better initialization," and we show that when sufficient local fine-tuning is allowed, a single global initialization can instead provide stronger personalization performance. This study redefines the role of global initialization under data and preference heterogeneity and provides practical training strategies that simultaneously satisfy global knowledge preservation and personalization.
