FuseFlow:一种面向稀疏深度学习的流数据流编译框架
FuseFlow: A Fusion-Centric Compilation Framework for Sparse Deep Learning on Streaming Dataflow
AI总结:
FuseFlow是一种用于稀疏深度学习的编译框架,通过融合稀疏操作提升模型效率,展示了融合粒度对模型性能的影响。
摘要:
随着深度学习模型的扩展,稀疏计算和专用数据流硬件已成为解决效率问题的强大解决方案。我们提出了FuseFlow,一种将用PyTorch编写的稀疏机器学习模型转换为融合稀疏数据流图的编译器,用于可重构数据流架构(RDAs)。FuseFlow是首个支持一般跨表达式融合的编译器。除了跨内核(表达式)的融合外,FuseFlow还支持诸如并行化、数据流排序和稀疏性阻塞等优化。它针对一个精确周期的数据流模拟器,用于融合策略的微架构分析。我们使用FuseFlow在四个实际机器学习应用中进行设计空间探索,显示完全融合(整个跨表达式融合所有端到端模型中的计算)并不总是稀疏模型的最佳选择——融合粒度取决于模型本身。FuseFlow还提供了一个启发式方法来识别和修剪次优配置。使用FuseFlow,我们实现了性能提升,包括在使用BigBird块稀疏注意力的GPT-3中,相比未融合基线的性能提升了约2.7倍。
英文摘要:
As deep learning models scale, sparse computation and specialized dataflow hardware have emerged as powerful solutions to address efficiency. We propose FuseFlow, a compiler that converts sparse machine learning models written in PyTorch to fused sparse dataflow graphs for reconfigurable dataflow architectures (RDAs). FuseFlow is the first compiler to support general cross-expression fusion of sparse operations. In addition to fusion across kernels (expressions), FuseFlow also supports optimizations like parallelization, dataflow ordering, and sparsity blocking. It targets a cycle-accurate dataflow simulator for microarchitectural analysis of fusion strategies. We use FuseFlow for design-space exploration across four real-world machine learning applications with sparsity, showing that full fusion (entire cross-expression fusion across all computation in an end-to-end model) is not always optimal for sparse models-fusion granularity depends on the model itself. FuseFlow also provides a heuristic to identify and prune suboptimal configurations. Using Fuseflow, we achieve performance improvements, including a ~2.7x speedup over an unfused baseline for GPT-3 with BigBird block-sparse attention.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。