AI 中文总结
FedGSA是用于差分隐私联邦LoRA的几何一致子空间聚合框架，通过格拉斯曼流形上的基不变子空间聚合减少更新扭曲，在GLUE等任务上较基线提升2.17%、2.27%且无额外隐私损失。
AI 中文摘要
低秩适配（LoRA）支持预训练语言模型的通信高效联邦微调，但将差分隐私（DP）集成到联邦LoRA中仍具挑战：对其两个低秩矩阵分别进行扰动和聚合会导致聚合不匹配及二次噪声项。现有方法通过冻结一个低秩矩阵缓解这些问题，但仍依赖欧几里得聚合，该聚合依赖基且可能扭曲全局更新。为解决此局限，我们提出FedGSA，一种用于差分隐私联邦LoRA的几何一致聚合框架。FedGSA将每个私有化的客户端更新表示为格拉斯曼流形上的基不变子空间。在每个通信轮次，客户端提取捕获主导更新方向的低维子空间，并将其编码为投影矩阵。服务器聚合这些表示以估计几何一致的全局更新子空间，并在其中重构全局LoRA因子，减少由基不对齐、隐私噪声和异构客户端更新导致的扭曲。我们证明FedGSA不会产生超出客户端DP训练的额外隐私损失，并在标准假设下建立其收敛性。在四个GLUE任务和一个语言生成基准上的实验表明，在不同隐私预算和数据异质性程度下，FedGSA均取得一致提升；尤其在ε=6和ε=3时，FedGSA较最强基线分别提升平均准确率2.17%和2.27%。
英文摘要
Low-Rank Adaptation (LoRA) enables communication-efficient federated fine-tuning of pretrained language models. However, integrating differential privacy (DP) into federated LoRA remains challenging: independently perturbing and aggregating its two low-rank matrices can cause aggregation mismatch and the quadratic noise term. Existing methods mitigate these issues by freezing one low-rank matrix but still rely on Euclidean aggregation, which is basis-dependent and may distort the global update. To address this limitation, we propose FedGSA, a geometry-consistent aggregation framework for differentially private federated LoRA. FedGSA represents each privatized client update as a basis-invariant subspace on the Grassmann manifold. In each communication round, clients extract low-dimensional subspaces capturing dominant update directions and encode them as projection matrices. The server aggregates these representations to estimate a geometry-consistent global update subspace and reconstructs the global LoRA factors within it, reducing distortion caused by basis misalignment, privacy noise, and heterogeneous client updates. We prove that FedGSA incurs no additional privacy loss beyond client-side DP training and establish its convergence under standard assumptions. Experiments on four GLUE tasks and a language generation benchmark demonstrate consistent improvements across privacy budgets and degrees of data heterogeneity. In particular, FedGSA improves average accuracy over the strongest baseline by 2.17% and 2.27% under $ε=6$ and $ε=3$, respectively.
