冗余分布式系统中的多样性/并行性权衡
Diversity/Parallelism Trade-off in Distributed Systems with Redundancy
摘要：
随着许多机器学习和其他算法的复杂性和数据需求的增加，分布式计算成为满足不断增长的计算和存储需求所必需的，因为它使得能够并行执行构成大型计算作业的较小任务。然而，任务服务时间的随机波动会导致执行时间较长的离散任务。以任务复制和擦除编码的形式，冗余提供了多样性，允许在仅执行冗余任务的子集时完成作业，从而消除对离散任务的依赖性。在资源受限的情况下（这里是固定数量的并行服务器），增加冗余会减少并行可用的资源。在本文中，我们的特点的多样性与并行性的权衡，并确定最佳的策略，复制，编码和分裂，最大限度地减少预期的作业完成时间。我们考虑了三种常见的服务时间分布，并建立了三个模型来描述这些分布与任务大小的缩放。我们发现，具有不同缩放模型的不同分布在不同的冗余水平下最佳地操作，因此可能需要非常不同的码率。
英文摘要：
As numerous machine learning and other algorithms increase in complexity and data requirements, distributed computing becomes necessary to satisfy the growing computational and storage demands, because it enables parallel execution of smaller tasks that make up a large computing job. However, random fluctuations in task service times lead to straggling tasks with long execution times. Redundancy, in the form of task replication and erasure coding, provides diversity that allows a job to be completed when only a subset of redundant tasks is executed, thus removing the dependency on the straggling tasks. In situations of constrained resources (here a fixed number of parallel servers), increasing redundancy reduces the available resources for parallelism. In this paper, we characterize the diversity vs. parallelism trade-off and identify the optimal strategy, among replication, coding and splitting, which minimizes the expected job completion time. We consider three common service time distributions and establish three models that describe scaling of these distributions with the task size. We find that different distributions with different scaling models operate optimally at different levels of redundancy, and thus may require very different code rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
