跨说话人编码网络用于多说话人语音识别
Cross-Speaker Encoding Network for Multi-Talker Speech Recognition
AI总结：
提出跨说话人编码网络，结合SIMO与SOT优势，在LibrispeechMix上较SIMO降低WER 8%，较SOT降低10%，高重叠语音降低16%。
摘要：
端到端多说话人语音识别作为一种直接转录多个说话人重叠语音的有效方法，已引起广泛关注。当前方法通常采用以下两种方式之一：1）具有分支编码器的单输入多输出（SIMO）模型，或2）基于注意力编码器-解码器架构并采用序列化输出训练（SOT）的单输入单输出（SISO）模型。在本工作中，我们提出了一种跨说话人编码（CSE）网络，通过聚合跨说话人表示来解决SIMO模型的局限性。此外，将CSE模型与SOT集成，以同时利用SIMO和SISO的优势，同时减轻其缺点。据我们所知，这项工作是为多说话人语音识别集成SIMO和SISO的早期尝试。在双说话人LibrispeechMix数据集上的实验表明，CES模型相比SIMO基线将词错误率（WER）降低了8%。与SOT模型相比，CSE-SOT模型在总体上将WER降低了10%，在高重叠语音上降低了16%。代码可在https://github.com/kjw11/CSEnet-ASR获取。
英文摘要：
End-to-end multi-talker speech recognition has garnered great interest as an effective approach to directly transcribe overlapped speech from multiple speakers. Current methods typically adopt either 1) single-input multiple-output (SIMO) models with a branched encoder, or 2) single-input single-output (SISO) models based on attention-based encoder-decoder architecture with serialized output training (SOT). In this work, we propose a Cross-Speaker Encoding (CSE) network to address the limitations of SIMO models by aggregating cross-speaker representations. Furthermore, the CSE model is integrated with SOT to leverage both the advantages of SIMO and SISO while mitigating their drawbacks. To the best of our knowledge, this work represents an early effort to integrate SIMO and SISO for multi-talker speech recognition. Experiments on the two-speaker LibrispeechMix dataset show that the CES model reduces word error rate (WER) by 8% over the SIMO baseline. The CSE-SOT model reduces WER by 10% overall and by 16% on high-overlap speech compared to the SOT model. Code is available at https://github.com/kjw11/CSEnet-ASR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
