arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.07555cs.DC
收藏

SerPyTor:一种用于持久执行的分布式上下文感知计算图执行框架

SerPyTor: A distributed context-aware computational graph execution framework for durable execution

Anuran Roy, Sridhar Raj S

AI总结:

本文提出 SerPyTor,一个轻量级、上下文感知的分布式计算图执行框架,以低设置开销和快速执行速度解决 Spark 集群在受限场景中上下文传播的不足。

摘要:

分布式计算始终是一个棘手的话题,尤其是在不同场景下存在各种需求的情况下。一种流行的解决方案是使用 Apache Spark,并配置多个系统组成集群。然而,Spark 集群的前置设置通常会带来额外的开销,这在受限场景中往往限制了其使用,尤其是在需要上下文传播的场景中。在本文中,我们探索了一种相对轻量级的计算图执行框架,该框架只需很少的设置且速度很快,同时具备上下文感知能力。

英文摘要:

Distributed computation is always a tricky topic to deal with, especially in context of various requirements in various scenarios. A popular solution is to use Apache Spark with a setup of multiple systems forming a cluster. However, the prerequisite setup for a Spark cluster often induces an additional overhead, often limiting usage in constrained scenarios, especially in scenarios requiring context propagation. In this paper, we explore a relatively lightweight computational graph execution framework requiring little setup and fast speeds, coupled with context awareness.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑