Fathom:现代深度学习方法的参考工作负载
Fathom: Reference Workloads for Modern Deep Learning Methods
摘要:
深度学习因其最近在挑战人工智能问题上的成功而受到普及。其主导地位的原因之一也是一个持续的挑战:对巨大计算能力的需求。硬件架构师已经提出了一系列有前途的想法,但到目前为止,大多数工作都集中在一些狭窄的应用领域的特定算法。虽然这些问题的特殊性并没有削弱这些办法,但显然需要更灵活的解决办法。我们认为第一步是检查整个深度学习社区的尖端模型的特征。因此,我们组装了Fathom:一个包含八个典型深度学习工作负载的集合。这些模型中的每一个都来自深度学习社区的开创性工作,从熟悉的Krizhevsky等人的深度卷积神经网络,Facebook的人工智能研究小组的更奇特的记忆网络。Fathom已在线发布,本文着重于了解每个模型的基本性能特征。我们使用一组围绕TensorFlow深度学习框架构建的应用程序级建模工具来分析Fathom工作负载的行为。我们提出了一个细分的时间花在哪里,我们的模型的性能配置文件之间的相似性,推理和训练的行为分析,以及并行性对扩展的影响。
英文摘要:
Deep learning has been popularized by its recent successes on challenging artificial intelligence problems. One of the reasons for its dominance is also an ongoing challenge: the need for immense amounts of computational power. Hardware architects have responded by proposing a wide array of promising ideas, but to date, the majority of the work has focused on specific algorithms in somewhat narrow application domains. While their specificity does not diminish these approaches, there is a clear need for more flexible solutions. We believe the first step is to examine the characteristics of cutting edge models from across the deep learning community. Consequently, we have assembled Fathom: a collection of eight archetypal deep learning workloads for study. Each of these models comes from a seminal work in the deep learning community, ranging from the familiar deep convolutional neural network of Krizhevsky et al., to the more exotic memory networks from Facebook's AI research group. Fathom has been released online, and this paper focuses on understanding the fundamental performance characteristics of each model. We use a set of application-level modeling tools built around the TensorFlow deep learning framework in order to analyze the behavior of the Fathom workloads. We present a breakdown of where time is spent, the similarities between the performance profiles of our models, an analysis of behavior in inference and training, and the effects of parallelism on scaling.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。