arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.01633cs.ITmath.IT
收藏

分布式线性可分离计算中计算与通信代价的权衡

On the Tradeoff Between Computation and Communication Costs for Distributed Linearly Separable Computation

Kai Wan, Hua Sun, Mingyue Ji, Giuseppe Caire

摘要:

本文研究分布式线性可分计算问题,它是现有的许多分布式计算问题如分布式梯度下降和分布式线性变换的推广。在这个问题中,主节点要求$N$分布式工作节点计算$K$数据集的线性可分函数,该函数是$K$消息的一组$K_c$线性组合(每个消息是一个数据集的函数)。我们将一些数据集分配给每个工作者,然后工作者计算相应的消息并返回这些消息的某个函数,这样主工作者就可以从$N$中的任何$N_r$的答案中恢复任务函数。在文献中,已经考虑了$K_c = 1$或计算成本最小的特定情况。在本文中,我们专注于一般情况(即,通用$K_c$和通用计算成本),并且旨在找到最小通信成本。我们首先提出了一个新的通信成本的约束下流行的循环分配(广泛认为在文献中),分配数据集的工人在一个循环的方式下的逆界。基于现有的分布式计算策略不能达到逆界的观点,提出了一种新的分布式计算方案.当$K_c$较大时,该算法对任何分配都是最优的;当工作者和数据集数目相等或$K_c$较小时,该算法对循环分配是最优的.此外,它是在循环分配的情况下,其余情况下的一个因素2阶最优。

英文摘要:

This paper studies the distributed linearly separable computation problem, which is a generalization of many existing distributed computing problems such as distributed gradient descent and distributed linear transform. In this problem, a master asks $N$ distributed workers to compute a linearly separable function of $K$ datasets, which is a set of $K_c$ linear combinations of $K$ messages (each message is a function of one dataset). We assign some datasets to each worker, which then computes the corresponding messages and returns some function of these messages, such that from the answers of any $N_r$ out of $N$ workers the master can recover the task function. In the literature, the specific case where $K_c = 1$ or where the computation cost is minimum has been considered. In this paper, we focus on the general case (i.e., general $K_c$ and general computation cost) and aim to find the minimum communication cost. We first propose a novel converse bound on the communication cost under the constraint of the popular cyclic assignment (widely considered in the literature), which assigns the datasets to the workers in a cyclic way. Motivated by the observation that existing strategies for distributed computing fall short of achieving the converse bound, we propose a novel distributed computing scheme for some system parameters. The proposed computing scheme is optimal for any assignment when $K_c$ is large and is optimal under cyclic assignment when the numbers of workers and datasets are equal or $K_c$ is small. In addition, it is order optimal within a factor of 2 under cyclic assignment for the remaining cases.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑