arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2212.11268cs.LGcs.ITeess.SPmath.IT
收藏

基于动态通信图的个性化分散多任务学习

Personalized Decentralized Multi-Task Learning Over Dynamic Communication Graphs

University of Maryland, College Park(马里兰大学帕克分校)

Matin Mortaheb, Sennur Ulukus

摘要:

分散式和联邦学习算法面临的最大挑战之一是数据异构性,特别是当用户想要学习特定任务时。即使在将个性化报头连接到共享网络(PF-MTL)时,使用分散算法聚合所有网络也会由于数据的异构性而导致性能下降。我们的算法使用交换梯度来自动计算任务之间的相关性,并动态调整通信图,以连接互利的任务,并隔离那些可能相互产生负面影响的任务。该算法提高了学习性能,并导致更快的收敛相比,所有客户端连接到彼此的情况下,无论他们的相关性。我们在合成高斯数据集和大规模名人属性(CelebA)数据集上进行实验。合成数据的实验表明,我们提出的方法是能够检测的任务是积极的和消极的相关性。此外,CelebA的实验结果表明,该方法可以产生比完全连接网络更快的训练结果。

英文摘要:

Decentralized and federated learning algorithms face data heterogeneity as one of the biggest challenges, especially when users want to learn a specific task. Even when personalized headers are used concatenated to a shared network (PF-MTL), aggregating all the networks with a decentralized algorithm can result in performance degradation as a result of heterogeneity in the data. Our algorithm uses exchanged gradients to calculate the correlations among tasks automatically, and dynamically adjusts the communication graph to connect mutually beneficial tasks and isolate those that may negatively impact each other. This algorithm improves the learning performance and leads to faster convergence compared to the case where all clients are connected to each other regardless of their correlations. We conduct experiments on a synthetic Gaussian dataset and a large-scale celebrity attributes (CelebA) dataset. The experiment with the synthetic data illustrates that our proposed method is capable of detecting tasks that are positively and negatively correlated. Moreover, the results of the experiments with CelebA demonstrate that the proposed method may produce significantly faster training results than fully-connected networks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑