CMT-LLM：利用大语言模型的上下文多说话人自动语音识别
CMT-LLM: Contextual Multi-Talker ASR Utilizing Large Language Models
AI总结：
针对传统多说话人ASR与上下文偏置分离处理性能受限的问题，提出CMT-LLM统一框架，结合预训练语音编码器与LLM，引入两阶段过滤算法优化偏置词注入，在复杂场景下稀有词识别与词错率表现优于传统方法。
摘要：
在实际应用中，自动语音识别（ASR）系统必须处理多个说话人的重叠语音，并识别技术术语等稀有词。传统方法分别处理多说话人ASR和上下文偏置，限制了复杂场景下的性能。我们提出一个统一框架，将多说话人重叠语音识别和上下文偏置整合为单个任务。我们的ASR方法整合了预训练语音编码器和大语言模型（LLM），采用优化的微调策略。我们还引入了两阶段过滤算法，可从大型偏置列表中高效识别相关稀有词，并将其纳入LLM的提示输入中，提升稀有词识别效果。实验表明，我们的方法优于传统上下文偏置方法，当偏置规模为1000时，在LibriMix上的词错误率（WER）为7.9%，在AMI SDM上为32.9%，证明了其在复杂语音场景中的有效性。
英文摘要：
In real-world applications, automatic speech recognition (ASR) systems must handle overlapping speech from multiple speakers and recognize rare words like technical terms. Traditional methods address multi-talker ASR and contextual biasing separately, limiting performance in complex scenarios. We propose a unified framework that combines multi-talker overlapping speech recognition and contextual biasing into a single task. Our ASR method integrates pretrained speech encoders and large language models (LLMs), using optimized finetuning strategies. We also introduce a two-stage filtering algorithm to efficiently identify relevant rare words from large biasing lists and incorporate them into the LLM's prompt input, enhancing rare word recognition. Experiments show that our approach outperforms traditional contextual biasing methods, achieving a WER of 7.9% on LibriMix and 32.9% on AMI SDM when the biasing size is 1,000, demonstrating its effectiveness in complex speech scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
