面向数据异构性下的隐私保护联邦提示调优：一种子空间分解专家方法
Towards Privacy-Preserving Federated Prompt Tuning under Data Heterogeneity: A Subspace-Decomposed Expert Approach
浏览论文内容
中文总结 AI 辅助
研究针对数据异构性下的隐私保护联邦提示调优问题，提出FedSEPT方法，采用子空间分解专家建模及实例感知专家融合技术，在11个异构基准实验中，于相同隐私约束下，该方法在局部适配和全局泛化间实现更好权衡。
中文摘要 AI 辅助
联邦提示调优（FPT）能够使用轻量级提示对视觉语言模型（VLM）进行协作式适配。现有方法常通过局部差分隐私（DP）下的拆分提示设计来解决异构性和隐私问题，结合用于全局转移的共享提示和用于局部适配的私有提示。但单一共享提示可能过度平滑多样的可转移知识，削弱个性化与泛化之间的平衡。多专家提示（MEP）能更好地捕捉这种多样性，但会扩大通信空间，增加DP噪声和通信成本，且使稳健的专家组合更困难。我们提出了FedSEPT，一种隐私保护的联邦子空间分解专家提示调优方法。具体而言，我们采用子空间分解专家建模（SEM），用共享低秩因子、固定公共基和私有残差对多个提示专家进行参数化，将通信和DP扰动限制在紧凑的因子空间，同时在公共坐标系中实现直接服务器聚合。我们还设计了实例感知专家融合（IEF），通过设备上的路由自适应地组合语义互补的专家，并使用缓存的特定专家文本特征进行高效的逻辑级融合。在11个异构基准上的大量实验表明，在相同隐私约束下，FedSEPT在局部适配和全局泛化之间实现了比强大基线更好的权衡。
英文摘要
Federated prompt tuning (FPT) enables collaborative adaptation of vision--language models (VLMs) using lightweight prompts. Existing methods often address heterogeneity and privacy through a split-prompt design under local differential privacy (DP), combining a shared prompt for global transfer with private prompts for local adaptation. However, a single shared prompt may over-smooth diverse transferable knowledge, weakening the balance between personalization and generalization. Multi-expert prompts (MEPs) can better capture this diversity, but enlarge the communicated space, increasing DP noise and communication cost while making robust expert composition more difficult. We propose FedSEPT, a privacy-preserving Fed}erated Subspace-decomposed Expert Prompt Tuning. Specifically, we employ Subspace-decomposed Expert Modeling (SEM) to parameterize multiple prompt experts with shared low-rank factors, a fixed public basis, and private residuals, thereby confining communication and DP perturbation to a compact factor space while enabling direct server aggregation in a common coordinate system. We further design Instance-aware Expert Fusion (IEF), which adaptively combines semantically complementary experts via on-device routing and performs efficient logit-level fusion using cached expert-specific text features. Extensive experiments on 11 heterogeneous benchmarks show that, under the same privacy constraints, FedSEPT achieves a better trade-off between local adaptation and global generalization than strong baselines.
发表机构
- School of Artificial Intelligence, Beihang University（北京航空航天大学人工智能学院）
- Center for the Applied Statistics, School of Statistics, Renmin University of China（中国人民大学统计学院应用统计中心）
- School of Computer Science & Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）
- School of Software, Beihang University（北京航空航天大学软件学院）
- School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）
机构由 AI 辅助整理，请以论文原文为准。
