适应性语音识别条件Whisper用于端到端多说话人语音识别
Adapting Diarization-Conditioned Whisper for End-to-End Multi-Talker Speech Recognition
AI总结：
本文提出一种基于Whisper的多说话人语音识别模型，通过联合解码和序列化输出提升多说话人场景下的识别性能。
摘要：
我们提出了一种基于语音识别条件Whisper的说话人属性（SA）模型，用于多说话人语音识别，该模型结合了目标说话人建模与序列化输出训练（SOT）。我们的方法利用语音条件Whisper（DiCoW）编码器提取目标说话人嵌入，这些嵌入被连接成单一表示并传递到共享解码器。这使模型能够将重叠语音作为序列化输出流进行转录，带有说话人标签和时间戳。与如DiCoW等针对目标说话人的ASR系统不同，我们的方法进行联合解码，使解码器能够同时考虑所有说话人的上下文。实验表明，该模型在现有SOT方法上表现更优，并在多说话人混合（如LibriMix）上超越了DiCoW。
英文摘要：
We propose a speaker-attributed (SA) Whisper-based model for multi-talker speech recognition that combines target-speaker modeling with serialized output training (SOT). Our approach leverages a Diarization-Conditioned Whisper (DiCoW) encoder to extract target-speaker embeddings, which are concatenated into a single representation and passed to a shared decoder. This enables the model to transcribe overlapping speech as a serialized output stream with speaker tags and timestamps. In contrast to target-speaker ASR systems such as DiCoW, which decode each speaker separately, our approach performs joint decoding, allowing the decoder to condition on the context of all speakers simultaneously. Experiments show that the model outperforms existing SOT-based approaches and surpasses DiCoW on multi-talker mixtures (e.g., LibriMix).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
