使用动态计算图的深度学习
Deep Learning with Dynamic Computation Graphs
摘要:
在图结构上进行计算的神经网络自然适合于各种领域的问题,包括自然语言(解析树)和化学信息学(分子图)。然而,由于计算图对于每个输入都具有不同的形状和大小,因此这种网络不直接支持批量训练或推理。它们也很难在流行的深度学习库中实现,这些库基于静态数据流图。我们引入了一种称为动态图形化的技术,它不仅将不同形状的不同输入图之间的操作批量化,而且还将单个输入图中的不同节点之间的操作批量化。该技术允许我们使用流行的库创建静态图,模拟任意形状和大小的动态计算图。我们进一步提出了一个高层次的组合块库,简化了动态图模型的创建。使用该库,我们展示了简洁和批量并行实现的各种型号的文献。
英文摘要:
Neural networks that compute over graph structures are a natural fit for problems in a variety of domains, including natural language (parse trees) and cheminformatics (molecular graphs). However, since the computation graph has a different shape and size for every input, such networks do not directly support batched training or inference. They are also difficult to implement in popular deep learning libraries, which are based on static data-flow graphs. We introduce a technique called dynamic batching, which not only batches together operations between different input graphs of dissimilar shape, but also between different nodes within a single input graph. The technique allows us to create static graphs, using popular libraries, that emulate dynamic computation graphs of arbitrary shape and size. We further present a high-level library of compositional blocks that simplifies the creation of dynamic graph models. Using the library, we demonstrate concise and batch-wise parallel implementations for a variety of models from the literature.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。