ADDS:用于鲁棒多方学习的自适应微分采样
ADDS: Adaptive Differentiable Sampling for Robust Multi-Party Learning
摘要:
分布式多方学习为在法律和实际约束条件下训练具有分散数据的联合模型提供了一种有效途径。然而,由于数据标签在参与者之间的倾斜分布和本地设备的计算瓶颈的泥潭,如何在各种场景中为客户端构建更小的定制模型,同时提供适用于中心模型的更新仍然是一个挑战。在本文中,我们提出了一种新的自适应微分采样框架(ADDS)的鲁棒性和通信效率的多方学习。受神经网络中的丢包思想的启发,本文提出了一种多方环境下的网络采样策略,将中心模型的不同节点分布到客户端进行更新,可区分的采样率允许每个客户端根据其私有数据分布从超网中提取最优的局部结构。该方法只需对现有的多方学习结构进行最小的修改,并且能够将所有节点的局部更新集成到超网中,提高了中心模型的鲁棒性。所提出的框架显着降低了本地计算和通信成本,同时加快了中央模型的收敛,正如我们通过对真实世界数据集的实验所证明的。
英文摘要:
Distributed multi-party learning provides an effective approach for training a joint model with scattered data under legal and practical constraints. However, due to the quagmire of a skewed distribution of data labels across participants and the computation bottleneck of local devices, how to build smaller customized models for clients in various scenarios while providing updates appliable to the central model remains a challenge. In this paper, we propose a novel adaptive differentiable sampling framework (ADDS) for robust and communication-efficient multi-party learning. Inspired by the idea of dropout in neural networks, we introduce a network sampling strategy in the multi-party setting, which distributes different subnets of the central model to clients for updating, and the differentiable sampling rates allow each client to extract optimal local architecture from the supernet according to its private data distribution. The approach requires minimal modifications to the existing multi-party learning structure, and it is capable of integrating local updates of all subnets into the supernet, improving the robustness of the central model. The proposed framework significantly reduces local computation and communication costs while speeding up the central model convergence, as we demonstrated through experiments on real-world datasets.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。