通过相邻Token合并加速Transducer
Accelerating Transducers through Adjacent Token Merging
AI总结:
提出相邻Token合并(A-ToMe)方法,通过合并高相似度相邻token减少时间步长,在LibriSpeech上减少57%token并提升70%GPU推理速度且无明显精度损失,同时适用于长篇ASR。
摘要:
近期的端到端自动语音识别(ASR)系统通常利用基于Transformer的声学编码器,以高帧率生成embedding。然而,由于自注意力的二次计算复杂度,这种设计效率低下,尤其是对于长语音信号。为了解决这一问题,我们提出了一种新方法——相邻Token合并(A-ToMe),该方法逐步合并键值之间相似度得分较高的相邻token。通过这种方式,可以减少总时间步长,并加速编码器和联合网络的推理。在LibriSpeech上的实验表明,我们的方法能够减少57%的token,并在GPU上将推理速度提高70%,且没有任何显著的精度损失。此外,我们证明A-ToMe也是减少长篇ASR中token的有效解决方案,其中输入语音由多个话语组成。
英文摘要:
Recent end-to-end automatic speech recognition (ASR) systems often utilize a Transformer-based acoustic encoder that generates embedding at a high frame rate. However, this design is inefficient, particularly for long speech signals due to the quadratic computation of self-attention. To address this, we propose a new method, Adjacent Token Merging (A-ToMe), which gradually combines adjacent tokens with high similarity scores between their key values. In this way, the total time step could be reduced, and the inference of both the encoder and joint network is accelerated. Experiments on LibriSpeech show that our method can reduce 57% of tokens and improve the inference speed on GPU by 70% without any notable loss of accuracy. Additionally, we demonstrate that A-ToMe is also an effective solution to reduce tokens in long-form ASR, where the input speech consists of multiple utterances.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。