对异构联邦学习中个性化微调的深入探讨
A Closer Look at Personalized Fine-Tuning in Heterogeneous Federated Learning
AI总结：
本文提出LP-FT方法，通过缓解特征扭曲来提升异构联邦学习中个性化与泛化能力的平衡，揭示了联邦特征扭曲现象并提供了部署鲁棒个性化的方法指导。
摘要：
联邦学习（FL）能够实现去中心化的模型训练，同时保护隐私，但其在平衡全局泛化能力和本地个性化之间面临挑战，因为客户端的数据分布并不相同。个性化微调（PFT）是一种流行的事后解决方案，它在本地对最终的全局模型进行微调，但常常会过拟合于倾斜的客户端分布或在领域转移时失败。我们提出将线性探测后接完整微调（LP-FT）这种以集中化策略缓解特征扭曲（Kumar等，2022）的方法应用于FL环境。通过在七个数据集和六个PFT变体上的系统评估，我们展示了LP-FT在平衡个性化和泛化能力方面的优越性。我们的分析揭示了联邦特征扭曲现象，即本地微调会破坏全局学习的特征，并从理论上阐明了LP-FT通过分阶段的参数更新来缓解这一问题的方式。我们进一步建立了条件（例如部分特征重叠、协变量-概念转移）下，LP-FT优于标准微调的情况，为在FL中部署鲁棒的个性化提供了可行的指导方针。
英文摘要：
Federated Learning (FL) enables decentralized, privacy-preserving model training but struggles to balance global generalization and local personalization due to non-identical data distributions across clients. Personalized Fine-Tuning (PFT), a popular post-hoc solution, fine-tunes the final global model locally but often overfits to skewed client distributions or fails under domain shifts. We propose adapting Linear Probing followed by full Fine-Tuning (LP-FT), a principled centralized strategy for alleviating feature distortion (Kumar et al., 2022), to the FL setting. Through systematic evaluation across seven datasets and six PFT variants, we demonstrate LP-FT's superiority in balancing personalization and generalization. Our analysis uncovers federated feature distortion, a phenomenon where local fine-tuning destabilizes globally learned features, and theoretically characterizes how LP-FT mitigates this via phased parameter updates. We further establish conditions (e.g., partial feature overlap, covariate-concept shift) under which LP-FT outperforms standard fine-tuning, offering actionable guidelines for deploying robust personalization in FL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
