arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1908.11338cs.PFcs.LG
收藏

TapirXLA:使用Tapir将Fork-Join算法嵌入到TensorFlow中的XLA算法中

TapirXLA: Embedding Fork-Join Parallelism into the XLA Compiler in TensorFlow Using Tapir

MIT Computer Science and Artificial Intelligence Laboratory(麻省理工学院计算机科学与人工智能实验室);MIT Lincoln Laboratory(麻省理工学院林肯实验室)

Tao B. Schardl, Siddharth Samsi

摘要:

这项工作引入了TapirXLA,这是TensorFlow的XLA编译器的替代品,它将递归fork-join并行嵌入到XLA的低级别代码表示中。机器学习应用程序依赖于高效的并行处理来实现性能,并且它们采用各种技术来提高性能,包括编译器技术。但是机器学习框架中的编译器缺乏对并行性的深入理解,导致它们因错过并行计算的优化而失去性能。这项工作研究如何Tapir,编译器中间表示(IR),嵌入到主流编译器IR的并行性,可以被纳入编译器的机器学习,以解决这个问题。TapirXLA修改了TensorFlow中的XLA编译器,以使用Tapir/LLVM编译器来优化低级并行计算。TapirXLA使用Tapir的fork-join并行表示在高级TensorFlow操作中编码并行性。TapirXLA还向编译器公开了线性代数库例程的实现,这些例程的并行操作使用Tapir的表示进行编码。我们比较了使用TapirXLA的TensorFlow与使用未修改的XLA编译器的TensorFlow的性能。在四个神经网络基准测试中,TapirXLA在四个CPU架构中将网络的并行运行时间加快了30%到100%的几何平均乘法因子。

英文摘要:

This work introduces TapirXLA, a replacement for TensorFlow's XLA compiler that embeds recursive fork-join parallelism into XLA's low-level representation of code. Machine-learning applications rely on efficient parallel processing to achieve performance, and they employ a variety of technologies to improve performance, including compiler technology. But compilers in machine-learning frameworks lack a deep understanding of parallelism, causing them to lose performance by missing optimizations on parallel computation. This work studies how Tapir, a compiler intermediate representation (IR) that embeds parallelism into a mainstream compiler IR, can be incorporated into a compiler for machine learning to remedy this problem. TapirXLA modifies the XLA compiler in TensorFlow to employ the Tapir/LLVM compiler to optimize low-level parallel computation. TapirXLA encodes the parallelism within high-level TensorFlow operations using Tapir's representation of fork-join parallelism. TapirXLA also exposes to the compiler implementations of linear-algebra library routines whose parallel operations are encoded using Tapir's representation. We compared the performance of TensorFlow using TapirXLA against TensorFlow using an unmodified XLA compiler. On four neural-network benchmarks, TapirXLA speeds up the parallel running time of the network by a geometric-mean multiplicative factor of 30% to 100%, across four CPU architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑