基于AutoML的端到端ASR模型的迭代压缩
Iterative Compression of End-to-End ASR Model using AutoML
摘要:
对设备上自动语音识别(ASR)系统的日益增长的需求导致了对开发自动模型压缩技术的新兴趣。过去的研究表明,基于AutoML的低秩分解(LRF)技术,当应用于端到端编码器-注意-解码器风格的ASR模型时,可以实现高达3.7倍的加速,优于费力的手动排序方法。然而,我们表明,目前的AutoML为基础的搜索技术只能工作到一定的压缩水平,超过这一点,他们无法产生可接受的字出错率(WER)的压缩模型。在这项工作中,我们提出了一种基于AutoML的迭代LRF方法,该方法可以在不降低WER的情况下实现超过5倍的压缩,从而推进了ASR压缩的最新技术。
英文摘要:
Increasing demand for on-device Automatic Speech Recognition (ASR) systems has resulted in renewed interests in developing automatic model compression techniques. Past research have shown that AutoML-based Low Rank Factorization (LRF) technique, when applied to an end-to-end Encoder-Attention-Decoder style ASR model, can achieve a speedup of up to 3.7x, outperforming laborious manual rank-selection approaches. However, we show that current AutoML-based search techniques only work up to a certain compression level, beyond which they fail to produce compressed models with acceptable word error rates (WER). In this work, we propose an iterative AutoML-based LRF approach that achieves over 5x compression without degrading the WER, thereby advancing the state-of-the-art in ASR compression.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。