INT8 Winograd加速,适用于部署在移动设备上的配备ASR的Conv1D型号
INT8 Winograd Acceleration for Conv1D Equipped ASR Models Deployed on Mobile Devices
摘要:
自动语音识别(ASR)模型的密集计算阻碍了它们在移动设备上的部署。在本文中,我们提出了一种新的量化Winograd优化流水线,它结合了量化和快速卷积,以实现有效的推理加速移动设备上的ASR模型。为了避免量化和Winograd卷积结合带来的信息损失,提出了一种范围尺度量化(RSQ)训练方法,以扩大量化的数值范围,并从高精度值中提取知识。此外,改进的Conv1D装备DFSMN(ConvDFSMN)模型的移动部署设计。我们在ConvDFSMN和Wav2letter模型上进行了大量的实验。结果表明,该管道可以有效地优化模型。特别是,Wav2letter在基于ARMv7的移动设备上实现了1.48倍的加速比,WER降低了约0.07%。
英文摘要:
The intensive computation of Automatic Speech Recognition (ASR) models obstructs them from being deployed on mobile devices. In this paper, we present a novel quantized Winograd optimization pipeline, which combines the quantization and fast convolution to achieve efficient inference acceleration on mobile devices for ASR models. To avoid the information loss due to the combination of quantization and Winograd convolution, a Range-Scaled Quantization (RSQ) training method is proposed to expand the quantized numerical range and to distill knowledge from high-precision values. Moreover, an improved Conv1D equipped DFSMN (ConvDFSMN) model is designed for mobile deployment. We conduct extensive experiments on both ConvDFSMN and Wav2letter models. Results demonstrate the models can be effectively optimized with the proposed pipeline. Especially, Wav2letter achieves 1.48* speedup with an approximate 0.07% WER decrease on ARMv7-based mobile devices.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。