CF-LoRA：用于联邦LoRA微调的解耦因子聚合与自适应感知客户端聚类
CF-LoRA: Decoupled Factor Aggregation and Adaptation-Aware Client Clustering for Federated LoRA Fine-Tuning
- State Key Laboratory for Novel Software Technology（计算机软件新技术国家重点实验室）
- School of Artificial Intelligence（人工智能学院）
- School of Electronic Science and Engineering（电子科学与工程学院）
- Nanjing University（南京大学）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
CF-LoRA通过解耦LoRA因子聚合与自适应感知客户端聚类，解决联邦LoRA微调中异构数据下的结构聚合和统计协作不匹配，在语言和视觉任务上取得最高平均准确率且通信高效。
AI中文摘要：
联邦LoRA微调能够在无需共享私有数据的情况下对预训练模型进行参数高效适配，但在异构客户端数据下存在两个根本性不匹配：一是由于独立平均LoRA因子而导致的结构性聚合不匹配，二是由于在差异显著的客户端上强制使用单一全局适配器而导致的统计协作不匹配。为解决这些问题，我们提出了CF-LoRA，一种结合了解耦因子聚合与自适应感知客户端聚类的聚类联邦LoRA微调框架。CF-LoRA首先学习全局共享的$A$因子，同时保留个性化的$B_i$因子，然后基于学习到的$B_i$因子的余弦相似度识别具有相似自适应模式的客户端，最后在冻结$A$因子的情况下执行簇内$B$因子聚合。通过解耦LoRA因子聚合，CF-LoRA保留了低秩结构并缓解了结构性聚合不匹配，而自适应感知聚类促进了具有相似自适应模式的客户端之间的协作，并减少了由统计异质性引起的负迁移。在RoBERTa和ViT上的四个语言任务和四个视觉数据集上的实验表明，CF-LoRA在两种模态下均取得了最高的平均准确率，同时每轮优化仅通信一个LoRA因子。
英文摘要：
Federated LoRA fine-tuning enables parameter-efficient adaptation of pre-trained models without sharing private data, but suffers from two fundamental mismatches under heterogeneous client data: a structural aggregation mismatch caused by independently averaging LoRA factors, and a statistical collaboration mismatch caused by enforcing a single global adapter across divergent clients. To address these issues, we propose CF-LoRA, a clustered federated LoRA fine-tuning framework that combines decoupled factor aggregation with adaptation-aware client clustering. CF-LoRA first learns a globally shared $A$ factor while retaining personalized $B_i$ factors, then identifies clients with similar adaptation patterns based on the cosine similarity of their learned $B_i$ factors, and finally performs intra-cluster $B$-factor aggregation with a frozen $A$ factor. By decoupling LoRA factor aggregation, CF-LoRA preserves the low-rank structure and mitigates the structural aggregation mismatch, while adaptation-aware clustering promotes collaboration among clients with similar adaptation patterns and reduces negative transfer caused by statistical heterogeneity. Experiments on four language tasks and four vision datasets with RoBERTa and ViT show that CF-LoRA achieves the highest average accuracy in both modalities while communicating only one LoRA factor per optimization round.
