基于Transducer的多说话人ASR的无对齐训练
Alignment-Free Training for Transducer-based Multi-Talker ASR
AI总结：
本文提出MT-RNNT-AFT无对齐训练方案，通过在转录开头附加说话人提示标记，使标准RNNT架构无需精确时间戳即可实现多说话人ASR，性能与最先进方法相当且训练过程大幅简化。
摘要：
扩展RNN Transducer（RNNT）以识别多说话人语音对于更广泛的自动语音识别（ASR）应用至关重要。多说话人RNNT（MT-RNNT）旨在不依赖昂贵的前端声源分离实现识别。MT-RNNT通常通过具有多个编码器或解码器的架构实现，或者通过将所有说话人的转录串行化为单一输出流来实现。第一种方法计算成本高，特别是由于需要多个编码器处理。相比之下，第二种方法涉及复杂的标签生成过程，需要从外部ASR系统获得混合语音中所有说话人说出的所有单词的准确时间戳。在本文中，我们提出了一种新颖的MT-RNNT无对齐训练方案（MT-RNNT-AFT），该方案采用标准RNNT架构。目标标签通过在转录开头附加对应于每个说话人的提示标记来创建，反映每个说话人在混合语音中出现的顺序。因此，MT-RNNT-AFT可以在不依赖准确对齐的情况下进行训练，并且只需一轮编码器处理即可识别所有说话人的语音。实验表明，MT-RNNT-AFT取得了与最先进的替代方案相当的性能，同时大大简化了训练过程。
英文摘要：
Extending the RNN Transducer (RNNT) to recognize multi-talker speech is essential for wider automatic speech recognition (ASR) applications. Multi-talker RNNT (MT-RNNT) aims to achieve recognition without relying on costly front-end source separation. MT-RNNT is conventionally implemented using architectures with multiple encoders or decoders, or by serializing all speakers' transcriptions into a single output stream. The first approach is computationally expensive, particularly due to the need for multiple encoder processing. In contrast, the second approach involves a complex label generation process, requiring accurate timestamps of all words spoken by all speakers in the mixture, obtained from an external ASR system. In this paper, we propose a novel alignment-free training scheme for the MT-RNNT (MT-RNNT-AFT) that adopts the standard RNNT architecture. The target labels are created by appending a prompt token corresponding to each speaker at the beginning of the transcription, reflecting the order of each speaker's appearance in the mixtures. Thus, MT-RNNT-AFT can be trained without relying on accurate alignments, and it can recognize all speakers' speech with just one round of encoder processing. Experiments show that MT-RNNT-AFT achieves performance comparable to that of the state-of-the-art alternatives, while greatly simplifying the training process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
