使用张量核单元加速归约和扫描
Accelerating Reduction and Scan Using Tensor Core Units
摘要:
在深度学习的推动下,矩阵乘法的专用处理器激增,称为TensorCore单元(TCU)。这些TCU能够在小矩阵(通常为4x 4或16 x16)上执行矩阵乘法,以加速深度学习工作负载中的卷积和递归神经网络。在本文中,我们利用NVIDIA的TCU来表示矩阵乘法的减少和扫描,并显示其优点-在程序简单性,效率和性能方面。我们的算法可以使用闲置的NVIDIA TCU,实现89%-98%的峰值内存复制带宽,并且比机器学习和科学应用中常见的最先进的小段大小方法快几个数量级(缩减高达100倍,扫描高达3倍)。我们的算法实现了这一点,同时降低了功耗高达22%的减少和16%的扫描。
英文摘要:
Driven by deep learning, there has been a surge of specialized processors for matrix multiplication, referred to as TensorCore Units (TCUs). These TCUs are capable of performing matrix multiplications on small matrices (usually 4x4 or 16x16) to accelerate the convolutional and recurrent neural networks in deep learning workloads. In this paper we leverage NVIDIA's TCU to express both reduction and scan with matrix multiplication and show the benefits -- in terms of program simplicity, efficiency, and performance. Our algorithm exercises the NVIDIA TCUs which would otherwise be idle, achieves 89%-98% of peak memory copy bandwidth, and is orders of magnitude faster (up to 100x for reduction and 3x for scan) than state-of-the-art methods for small segment sizes -- common in machine learning and scientific applications. Our algorithm achieves this while decreasing the power consumption by up to 22% for reduction and16%for scan.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。