arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2304.11524cs.AI
收藏

通过梯度调制实现面向异构文本摘要的个性化联邦学习

Personalized Federated Learning via Gradient Modulation for Heterogeneous Text Summarization

Ping An Technology (Shenzhen) Co., Ltd.(平安科技(深圳)有限公司)

Rongfeng Pan, Jianzong Wang, Lingwei Kong, Zhangcheng Huang, Jing Xiao

AI总结:

针对异构文本摘要中的隐私保护与本地数据分布差异问题,提出带差分隐私的动态梯度适配器 FedSUMM,以改善个性化联邦学习的本地参数、收敛速度和摘要性能。

摘要:

文本摘要对于信息聚合至关重要,并且需要大量训练数据。然而,对数据隐私和安全的担忧限制了数据收集与模型训练。为消除这一担忧,我们提出一种联邦学习文本摘要方案,使用户能够以协作学习的方式共享全局模型,而无需共享原始数据。个性化联邦学习(PFL)在优化全局模型的过程中平衡个性化与泛化能力,以指导本地模型的训练。但是,多个本地数据具有不同的语义和上下文分布,这可能导致本地模型学习到偏离的语义和上下文信息。本文提出 FedSUMM,这是一种动态梯度适配器,用于为本地模型提供更合适的本地参数。同时,FedSUMM 使用差分隐私来防止分布式训练过程中的参数泄露。实验结果证明,在面向特定任务文本摘要的 PFL 算法上,FedSUMM 能够实现更快的模型收敛,并且该方法在文本摘要的不同优化指标上取得了优越性能。

英文摘要:

Text summarization is essential for information aggregation and demands large amounts of training data. However, concerns about data privacy and security limit data collection and model training. To eliminate this concern, we propose a federated learning text summarization scheme, which allows users to share the global model in a cooperative learning manner without sharing raw data. Personalized federated learning (PFL) balances personalization and generalization in the process of optimizing the global model, to guide the training of local models. However, multiple local data have different distributions of semantics and context, which may cause the local model to learn deviated semantic and context information. In this paper, we propose FedSUMM, a dynamic gradient adapter to provide more appropriate local parameters for local model. Simultaneously, FedSUMM uses differential privacy to prevent parameter leakage during distributed training. Experimental evidence verifies FedSUMM can achieve faster model convergence on PFL algorithm for task-specific text summarization, and the method achieves superior performance for different optimization metrics for text summarization.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑