arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.05680cs.ARcs.AIcs.LG
收藏

EXION:利用扩散模型的迭代间与迭代内输出稀疏性

EXION: Exploiting Inter- and Intra-Iteration Output Sparsity for Diffusion Models

KAIST(韩国科学技术院)

Jaehoon Heo, Adiwena Putra, Jieon Yoon, Sungwoong Yune, Hangyeol Lee, Ji-Hoon Kim, Joo-Young Kim

AI总结:

本文提出软硬件协同设计的扩散模型加速器 EXION,通过 FFN-Reuse、两步前导1预测与 ConMerge 利用迭代间和迭代内输出稀疏性,在不损失精度的同时大幅提升性能与能效。

摘要:

在过去几年中,扩散模型已成为新颖的 AI 解决方案,可根据文本提示生成多样的多模态输出。尽管其能力强大,但由于迭代式架构,它们面临计算方面的挑战,例如过高的延迟和能耗。虽然以往专注于 Transformer 加速的工作可以被应用,但扩散模型的迭代本质仍未得到解决。在本文中,我们提出 EXION,这是首个通过利用扩散模型中独特的迭代间和迭代内输出稀疏性来解决计算挑战的软硬件协同设计扩散加速器。为此,我们提出两项软件级优化。首先,我们引入 FFN-Reuse 算法,该算法识别并跳过不同迭代之间 FFN 层中的冗余计算(迭代间稀疏性)。其次,我们使用改进的 eager prediction 方法,该方法采用两步前导 1 检测来准确预测注意力分数,从而跳过一次迭代内不必要的计算(迭代内稀疏性)。我们还引入名为 ConMerge 的新型数据压缩机制,它能够通过将稀疏矩阵浓缩并合并为紧凑形式来提高硬件利用率。最后,EXION 拥有支持上述稀疏化算法的专用硬件架构,可将高输出稀疏性转化为更高的能效和性能。为验证 EXION 的可行性,我们首先证明它对各种类型的多模态扩散模型的精度没有影响。随后,我们在服务器级和边缘级两种设置下实例化 EXION,并将其性能与规格相近的 GPU 进行比较。我们的评估表明,与服务器 GPU 相比,EXION 在性能和能效上分别实现了 3.2–379.3 倍和 45.1–3067.6 倍的显著提升;与边缘 GPU 相比,则分别实现了 42.6–1090.9 倍和 196.9–4668.2 倍的提升。

英文摘要:

Over the past few years, diffusion models have emerged as novel AI solutions, generating diverse multi-modal outputs from text prompts. Despite their capabilities, they face challenges in computing, such as excessive latency and energy consumption due to their iterative architecture. Although prior works specialized in transformer acceleration can be applied, the iterative nature of diffusion models remains unresolved. In this paper, we present EXION, the first SW-HW co-designed diffusion accelerator that solves the computation challenges by exploiting the unique inter- and intra-iteration output sparsity in diffusion models. To this end, we propose two SW-level optimizations. First, we introduce the FFN-Reuse algorithm that identifies and skips redundant computations in FFN layers across different iterations (inter-iteration sparsity). Second, we use a modified eager prediction method that employs two-step leading-one detection to accurately predict the attention score, skipping unnecessary computations within an iteration (intra-iteration sparsity). We also introduce a novel data compaction mechanism named ConMerge, which can enhance HW utilization by condensing and merging sparse matrices into compact forms. Finally, it has a dedicated HW architecture that supports the above sparsity-inducing algorithms, translating high output sparsity into improved energy efficiency and performance. To verify the feasibility of the EXION, we first demonstrate that it has no impact on accuracy in various types of multi-modal diffusion models. We then instantiate EXION in both server- and edge-level settings and compare its performance against GPUs with similar specifications. Our evaluation shows that EXION achieves dramatic improvements in performance and energy efficiency by 3.2-379.3x and 45.1-3067.6x compared to a server GPU and by 42.6-1090.9x and 196.9-4668.2x compared to an edge GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑