用于语音识别的高效流式大语言模型
Efficient Streaming LLM for Speech Recognition
AI总结：
针对现有大语言模型语音识别方法处理长流式音频泛化差、计算效率低的问题，提出线性扩展的流式语音识别模型SpeechLLM-XL，通过分块处理与有限注意力窗口降低成本，在基准测试上表现优异且长语音无质量下降。
摘要：
近期研究表明，为大语言模型提供音频编码提示可解锁语音识别能力，但现有技术无法高效扩展，尤其在处理长流式音频输入时表现不佳——它们不仅在超出训练时见过的音频长度时泛化能力差，还因注意力的二次计算成本导致计算效率低下。\n本研究提出SpeechLLM-XL，一种用于流式语音识别的线性扩展纯解码器模型。我们使用可配置的分块处理音频，采用有限注意力窗口以降低计算量，每个音频块的文本标记会自回归生成直至预测出EOS。训练期间，我们利用从编码器输出估计的CTC强制对齐，将转录文本分割为多个块。块大小为1.28秒的SpeechLLM-XL在LibriSpeech的test clean/other测试集上分别取得2.7%和6.7%的词错误率（WER），且在长度为训练 utterance 10倍的长语音上未出现质量下降。
英文摘要：
Recent works have shown that prompting large language models with audio encodings can unlock speech recognition capabilities. However, existing techniques do not scale efficiently, especially while handling long form streaming audio inputs -- not only do they extrapolate poorly beyond the audio length seen during training, but they are also computationally inefficient due to the quadratic cost of attention. In this work, we introduce SpeechLLM-XL, a linear scaling decoder-only model for streaming speech recognition. We process audios in configurable chunks using limited attention window for reduced computation, and the text tokens for each audio chunk are generated auto-regressively until an EOS is predicted. During training, the transcript is segmented into chunks, using a CTC forced alignment estimated from encoder output. SpeechLLM-XL with 1.28 seconds chunk size achieves 2.7%/6.7% WER on LibriSpeech test clean/other, and it shows no quality degradation on long form utterances 10x longer than the training utterances.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
