arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.04201cs.LG
收藏

MrTF:用于直推式联邦学习的模型精炼框架

MrTF: Model Refinery for Transductive Federated Learning

Nanjing University(南京大学);Nanjing University of Science and Technology(南京理工大学);The Hong Kong Polytechnic University(香港理工大学)

Xin-Chun Li, Yang Yang, De-Chuan Zhan

AI总结:

针对联邦学习忽略待推断数据的问题,提出直推式联邦学习范式MrTF,利用测试样本精炼聚合模型,结合稳定教师、修正蒸馏和聚类标签精炼技术,提升推断性能。

摘要:

我们考虑一个真实世界场景:一个新建立的试点项目需要在隐私保护政策下,借助其他参与方的帮助,对新收集的数据进行推断。当前的联邦学习(FL)范式致力于解决数据异质性(data heterogeneity)问题,但未考虑待推断的数据。我们提出了一种新的学习范式,称为直推式联邦学习(transductive federated learning, TFL),以同时考虑待推断数据的结构信息。一方面,服务器可以利用预先可用的测试样本对聚合模型进行精炼,以实现稳健的模型融合,从而解决FL中的数据异质性挑战。另一方面,精炼过程将测试样本纳入训练,能够以直推方式生成更好的预测。我们提出了若干技术,包括稳定教师(stabilized teachers)、修正蒸馏(rectified distillation)和聚类标签精炼(clustered label refinery),以促进模型精炼过程。大量实验研究验证了所提出的用于直推式联邦学习的模型精炼框架(MrTF)的优越性。源代码可在 https://github.com/lxcnju/MrTF 获取。

英文摘要:

We consider a real-world scenario in which a newly-established pilot project needs to make inferences for newly-collected data with the help of other parties under privacy protection policies. Current federated learning (FL) paradigms are devoted to solving the data heterogeneity problem without considering the to-be-inferred data. We propose a novel learning paradigm named transductive federated learning (TFL) to simultaneously consider the structural information of the to-be-inferred data. On the one hand, the server could use the pre-available test samples to refine the aggregated models for robust model fusion, which tackles the data heterogeneity problem in FL. On the other hand, the refinery process incorporates test samples into training and could generate better predictions in a transductive manner. We propose several techniques including stabilized teachers, rectified distillation, and clustered label refinery to facilitate the model refinery process. Abundant experimental studies verify the superiorities of the proposed \underline{M}odel \underline{r}efinery framework for \underline{T}ransductive \underline{F}ederated learning (MrTF). The source code is available at \url{https://github.com/lxcnju/MrTF}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑