笛卡尔积实现的动态系统编码计算

中文摘要 AI 辅助

本文研究了动态系统中的编码分布式计算（CDC），在该系统中，工作节点可能离开，新的集群可能加入。幸存工作节点的缓存及其现有的Reduce分配保持不变，而到达集群带来的存储则被投入使用。与针对线性函数的弹性计算以及需要在所有用户间进行放置的动态编码缓存相比，本文考虑的模型在部分固定、部分新增的放置方案下，适用于通用的MapReduce任务。所提出的方案构建了集群级放置传递数组（PDA），并通过笛卡尔积将它们耦合。一种传递感知规则重新分配被遗弃的Reduce函数，而一种广义通信PDA允许甚至不持有Reduce函数的工作节点作为编码发射器。对于任意可行的断连情况，推导出了精确的负载。针对即时可解码的XOR多播的文件级逆命题表明，在没有断连的情况下，当新集群到达时，该方案在此类多播类中与最优方案的差距在2倍以内；在其他情况下，即使基准在所有可行的非编码放置方案上进行优化，差距也在4倍以内。

英文摘要

This paper studies coded distributed computing (CDC) in a dynamic system in which workers may depart and new clusters may join. The caches of the surviving workers and their existing Reduce assignments stay untouched, while the storage brought by arriving clusters is put to use. In contrast to elastic computing, which targets linear functions, and to dynamic coded caching, which requires placement across all users, the model considered here accommodates general MapReduce tasks under a placement that is partly fixed and partly new. The proposed scheme builds cluster-wise placement delivery arrays (PDAs) and couples them through a Cartesian product. A delivery-aware rule reassigns the abandoned Reduce functions, and a generalized communication PDA allows even workers that hold no Reduce function to serve as coded transmitters. For arbitrary feasible disconnections, the exact load is derived. A file-wise converse for instantly decodable XOR multicasts shows that, in the absence of disconnections, the scheme lies within a factor of two of the best scheme in this multicast class when new clusters arrive, and within a factor of four otherwise, even when the benchmark optimizes over all feasible uncoded placements.

发表机构

• ShanghaiTech University（上海科技大学）
• Guangxi Normal University（广西师范大学）

机构由 AI 辅助整理，请以论文原文为准。