通过大语言模型集成理解零样本罕见词识别的改进
Understanding Zero-shot Rare Word Recognition Improvements Through LLM Integration
AI总结：
本研究通过集成大语言模型与语音识别系统，在190,000小时数据集上验证了LLM-ASR架构在零样本罕见词识别上优于传统模型，并揭示了LLM对罕见词错误率的改进及适配器和高质量数据的关键作用。
摘要：
在本研究中，我们探讨了将大语言模型（LLM）与自动语音识别（ASR）系统集成的方法，特别聚焦于提升罕见词识别性能。利用一个主要来源于YouTube、经Whisper V3伪标签预处理的190,000小时数据集，我们证明在大型数据集上训练后，LLM-ASR架构在零样本罕见词识别任务上优于传统的Zipformer-Transducer模型。我们的分析揭示，LLM对罕见词错误率（R-WER）的改进贡献显著，而语音编码器主要决定整体转录性能（正字法词错误率O-WER和归一化词错误率N-WER）。通过广泛的消融研究，我们强调了适配器集成在将语音编码器输出与LLM的语言能力对齐中的重要性。此外，我们强调了高质量标注数据在实现最优性能中的关键作用。这些发现为基于LLM的ASR架构之间的协同作用提供了宝贵的见解，为未来大规模基于LLM的语音识别系统的进步铺平了道路。
英文摘要：
In this study, we investigate the integration of a large language model (LLM) with an automatic speech recognition (ASR) system, specifically focusing on enhancing rare word recognition performance. Using a 190,000-hour dataset primarily sourced from YouTube, pre-processed with Whisper V3 pseudo-labeling, we demonstrate that the LLM-ASR architecture outperforms traditional Zipformer-Transducer models in the zero-shot rare word recognition task, after training on a large dataset. Our analysis reveals that the LLM contributes significantly to improvements in rare word error rate (R-WER), while the speech encoder primarily determines overall transcription performance (Orthographic Word Error Rate, O-WER, and Normalized Word Error Rate, N-WER). Through extensive ablation studies, we highlight the importance of adapter integration in aligning speech encoder outputs with the LLM's linguistic capabilities. Furthermore, we emphasize the critical role of high-quality labeled data in achieving optimal performance. These findings provide valuable insights into the synergy between LLM-based ASR architectures, paving the way for future advancements in large-scale LLM-based speech recognition systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
