arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.01169cs.CVcs.AI
收藏

通过移除GELU激活实现整数SWIN Transformer的更快推理

Faster Inference of Integer SWIN Transformer by Removing the GELU Activation

McGill University(麦吉尔大学)

Mohammadreza Tayaranian, Seyyed Hasan Mozafari, James J. Clark, Brett Meyer, Warren Gross

AI总结:

针对SWIN transformer难以全整数量化、推理速度慢的问题,提出用ReLU替换GELU激活并结合迭代知识蒸馏补精度,实现了量化模型推理延迟至少11%的提升,且ImageNet精度下降不足0.5%。

摘要:

SWIN transformer是一种性能卓越的视觉Transformer模型,在图像分类任务中拥有当前最优的准确率。尽管取得了成功,但其独特的架构导致它与同类深度神经网络相比推理速度更慢。对模型进行整数量化是提升其推理延迟的方法之一,但现有最优方案尚无法实现模型的全量化。在本研究中,我们通过移除Swin Transformer中与GELU激活相关的浮点运算,进一步优化了现有最优方法的推理延迟。此前的研究提出用线性近似函数替代非整数运算,而我们提出用ReLU激活替换GELU。与现有方法相比,ReLU的优势在于内存占用低、计算复杂度小。我们采用迭代知识蒸馏来弥补GELU替换为ReLU带来的精度损失。我们对移除GELU的SWIN transformer进行了量化,实验结果表明,在NVIDIA RTX 4090 GPU上,我们可将量化后SWIN transformer的推理延迟提升至少11%,同时在ImageNet评估数据集上的精度下降控制在0.5%以内。

英文摘要:

SWIN transformer is a prominent vision transformer model that has state-of-the-art accuracy in image classification tasks. Despite this success, its unique architecture causes slower inference compared with similar deep neural networks. Integer quantization of the model is one of the methods used to improve its inference latency. However, state-of-the-art has not been able to fully quantize the model. In this work, we improve upon the inference latency of the state-of-the-art methods by removing the floating-point operations, which are associated with the GELU activation in Swin Transformer. While previous work proposed to replace the non-integer operations with linear approximation functions, we propose to replace GELU with ReLU activation. The advantage of ReLU over previous methods is its low memory and computation complexity. We use iterative knowledge distillation to compensate for the lost accuracy due to replacing GELU with ReLU. We quantize our GELU-less SWIN transformer and show that on an RTX 4090 NVIDIA GPU we can improve the inference latency of the quantized SWIN transformer by at least $11\%$ while maintaining an accuracy drop of under $0.5\%$ on the ImageNet evaluation dataset.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑