即时动态批处理
Just-in-Time Dynamic-Batching
摘要:
批处理是深度学习框架中提高计算效率的基本技术。虽然用于具有静态前馈计算图的模型的批处理是直接实现的,但是由于跨样本的可变计算图结构,用于诸如语法树或社交网络图的动态计算图的并行化是具有挑战性的。通过对Tree-LSTM模型的仿真和分析,我们展示了动态映射中图分析时间和映射有效性之间的关键权衡。基于这一发现,我们提出了一个动态的编译方法作为扩展MXNet Gluon的即时编译(JIT)框架。我们的经验表明,我们的方法在一个常见的动态工作负载上产生了高达6.25倍的速度,这是一个用于语义相关性任务的树-LSTM模型。
英文摘要:
Batching is an essential technique to improve computation efficiency in deep learning frameworks. While batch processing for models with static feed-forward computation graphs is straightforward to implement, batching for dynamic computation graphs such as syntax trees or social network graphs is challenging due to variable computation graph structure across samples. Through simulation and analysis of a Tree-LSTM model, we show the key trade-off between graph analysis time and batching effectiveness in dynamic batching. Based on this finding, we propose a dynamic batching method as an extension to MXNet Gluon's just-in-time compilation (JIT) framework. We show empirically that our method yields up to 6.25 times speed-up on a common dynamic workload, a tree-LSTM model for the semantic relatedness task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。