arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.09436cs.DCcs.PL
收藏

PaSh:轻触式数据并行Shell处理

PaSh: Light-touch Data-Parallel Shell Processing

Nikos Vasilakis, Konstantinos Kallas, Konstantinos Mamouras, Achilleas Benetopoulos, Lazar Cvetković

摘要:

本文介绍了并行化POSIX shell脚本的系统{ \scshape PaSh}。给定一个脚本,{ \scshape PaSh}将其转换为一个并行图,执行一系列保留语义的程序转换以暴露并行性,然后将该并行图转换回一个脚本--该脚本添加POSIX结构以显式地指导并行性,并与{ \scshape PaSh}耦合--提供了{ \scshape Unix}感知的运行时原语,以解决与性能和正确性相关的问题。轻量级注释语言允许命令开发人员表达关于其命令的关键并行性属性。对POSIX和GNU命令(两个大型且常用的组)的并行性研究指导了{ \scshape PaSh}使用的注释语言和优化的聚合器库。最后,{ \scshape PaSh}的{ \scshape PaSh}对44个未修改的{ \scshape Unix}脚本的广泛评估显示出显著的加速($0.89$ -- $61.1\times$,avg:$6.7\times$),这源于其程序转换和运行时原语的组合。

英文摘要:

This paper presents {\scshape PaSh}, a system for parallelizing POSIX shell scripts. Given a script, {\scshape PaSh} converts it to a dataflow graph, performs a series of semantics-preserving program transformations that expose parallelism, and then converts the dataflow graph back into a script -- one that adds POSIX constructs to explicitly guide parallelism coupled with {\scshape PaSh}-provided {\scshape Unix}-aware runtime primitives for addressing performance- and correctness-related issues. A lightweight annotation language allows command developers to express key parallelizability properties about their commands. An accompanying parallelizability study of POSIX and GNU commands -- two large and commonly used groups -- guides the annotation language and optimized aggregator library that {\scshape PaSh} uses. Finally, {\scshape PaSh}'s {\scshape PaSh}'s extensive evaluation over 44 unmodified {\scshape Unix} scripts shows significant speedups ($0.89$--$61.1\times$, avg: $6.7\times$) stemming from the combination of its program transformations and runtime primitives.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑