VENOM：一种释放稀疏张量核心性能的向量化N:M格式
VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores
AI总结：
针对NVIDIA稀疏张量核心仅支持2:4稀疏格式、稀疏比受限的问题，研究提出V:N:M向量化格式与Spatha高性能稀疏库，配合二阶剪枝技术，实现高稀疏比下的大幅加速与极小精度损失。
摘要：
深度学习模型的日益成功与规模扩张要求更高的计算效率与能效。稀疏化既能缩小模型体积，又能提升计算效率，且专用加速硬件正逐步普及。然而，要高效利用稀疏特性，需要内核实现、剪枝算法与存储格式的协同，才能发挥专用稀疏向量单元的硬件支持能力，NVIDIA的稀疏张量核心（Sparse Tensor Cores, SPTCs）便是这类单元的代表，其理论上可实现2倍加速。但SPTCs仅支持2:4格式，将可达到的稀疏比限制在50%。我们提出V:N:M格式，可在SPTCs上运行任意N:M比例的稀疏运算。为高效利用该格式，我们设计了Spatha——一款面向深度学习例程的高性能稀疏库。实验表明，Spatha相较于cuBLAS最高可实现37倍加速。我们还验证了一种二阶剪枝技术，该技术配合V:N:M格式可实现高稀疏比，且在现代Transformer模型中精度损失极小甚至无损失。
英文摘要：
The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2x speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparse-library for DL routines. We show that Spatha achieves up to 37x speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
