arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2403.19792cs.LGcs.AIcs.CRcs.DC
收藏

MAPL:模型无关的点对点学习

MAPL: Model Agnostic Peer-to-peer Learning

Delft University of Technology(代尔夫特理工大学);ENSTA Paris(巴黎国立高等先进技术学校);Institut Polytechnique de Paris(巴黎理工学院);Shell Global Solutions International B.V.(壳牌全球解决方案国际有限公司)

Sayak Mukherjee, Andrea Simonetto, Hadi Jamali-Rad

AI总结:

MAPL通过点对点通信同时学习异构个性化模型和协作图,利用对比损失和去中心化协作图学习,在不依赖中央服务器的情况下取得优于集中式模型无关方法的性能。

摘要:

在去中心化环境中实现异构客户端之间的有效协作,是文献中一个尚未被充分探索的方向。为了从结构上解决这一问题,我们提出了模型无关的点对点学习(简称 MAPL),这是一种通过相邻客户端之间的点对点通信同时学习异构个性化模型和协作图的新方法。MAPL 由两个主要模块组成:(i)本地级个性化模型学习(PML),利用客户端内部和客户端之间的对比损失组合;(ii)网络范围去中心化协作图学习(CGL),基于本地任务相似性以保护隐私的方式动态细化协作权重。我们的大量实验证明了 MAPL 的有效性,并且与集中式模型无关的对应方法相比,其性能具有竞争力(在大多数情况下更优),且不依赖任何中央服务器。我们的代码已公开,可通过以下链接访问:https://github.com/SayakMukherjee/MAPL

英文摘要:

Effective collaboration among heterogeneous clients in a decentralized setting is a rather unexplored avenue in the literature. To structurally address this, we introduce Model Agnostic Peer-to-peer Learning (coined as MAPL) a novel approach to simultaneously learn heterogeneous personalized models as well as a collaboration graph through peer-to-peer communication among neighboring clients. MAPL is comprised of two main modules: (i) local-level Personalized Model Learning (PML), leveraging a combination of intra- and inter-client contrastive losses; (ii) network-wide decentralized Collaborative Graph Learning (CGL) dynamically refining collaboration weights in a privacy-preserving manner based on local task similarities. Our extensive experimentation demonstrates the efficacy of MAPL and its competitive (or, in most cases, superior) performance compared to its centralized model-agnostic counterparts, without relying on any central server. Our code is available and can be accessed here: https://github.com/SayakMukherjee/MAPL

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑