arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.02371cs.PL
收藏

在用户可调度语言中推动张量加速器超越MatMul

Pushing Tensor Accelerators Beyond MatMul in a User-Schedulable Language

University of Washington(华盛顿大学);Adobe

Yihong Zhang, Derek Gerstmann, Andrew Adams, Maaz Bin Safeer Ahmad

AI总结:

本文提出基于Halide语言的编译器技术,使开发者能利用张量加速器超越传统MatMul,实现图像处理等应用的加速。

摘要:

如今,张量加速器在现代CPU和GPU中的计算资源中占据越来越大的份额。然而,它们难以编程,导致开发者倾向于使用供应商提供的内核库来支持张量加速器。因此,张量加速器的使用被限制在提供的接口中,主要设计用于传统的机器学习和科学计算工作负载。在本文中,我们展示了张量加速器可以提高超越简单变种的MatMul的应用性能。例如,许多图像处理流水线实际上是矩阵上的线性变换,因此可以利用这种专用硬件。然而,这仍然受到编程张量加速器的困难的阻碍。我们通过编译器技术来解决这个问题。我们使用Halide用户可调度语言,简洁地将操作表示为Halide算法。为此,我们实现了一个基于等式饱和的灵活张量指令选择器。张量指令选择器支持CPU和GPU连接的张量加速器,并与现有的调度操作(例如生产者-消费者融合)一起工作。共同,这使开发者能够在几十行代码中编写多样化的加速器利用应用。使用我们的系统,我们展示了张量加速器超越传统领域的潜力。我们实现了几个图像处理流水线(例如,滤波、重采样和去噪)并在系统中评估它们与非加速器利用基线。我们显示这些流水线可以实现显著的加速。例如,利用Nvidia RTX 4070 GPU上的Tensor Cores,下采样程序的加速率为6.1倍。

英文摘要:

Tensor accelerators now represent a growing share of compute resources in modern CPUs and GPUs. However, they are hard to program, leading developers to use vendor-provided kernel libraries that support tensor accelerators. As a result, the usage of tensor accelerators is limited to the provided interface, mainly designed for traditional ML and scientific computing workloads. In this paper, we show that tensor accelerators can improve the performance of applications beyond simple variants of MatMul. For example, many image processing pipelines are linear transformations over matrices in disguise and can therefore utilize such specialized hardware. This is nonetheless hindered by the difficulties in programming tensor accelerators. We tackle this problem with compiler-based techniques. We use the Halide user-schedulable language and express operations as Halide algorithms succinctly. To this end, we implement a flexible tensor instruction selector based on equality saturation. The tensor instruction selector supports both CPU- and GPU-attached tensor accelerators and works with existing scheduling operations (e.g., producer-consumer fusion). Together, this enables developers to write diverse accelerator-leveraging applications in a few dozen lines. Using our system, we demonstrate the potential of tensor accelerators beyond their traditional domains. We implement several image processing pipelines (e.g., filtering, resampling, and denoising) in our system and evaluate them against non-accelerator-leveraging baselines. We show that these pipelines can achieve significant speedups. For example, a downsampling routine is sped up by $6.1\times$ by utilizing Tensor Cores on an Nvidia RTX 4070 GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑