arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2104.01882eess.AS
收藏

多说话人会话ASR的说话人条件声学建模

Speaker conditioned acoustic modeling for multi-speaker conversational ASR

Srikanth Raj Chetupalli, Sriram Ganapathy

摘要:

在本文中,我们提出了一种新的方法,从单通道语音记录的语音对话与自然扬声器重叠的转录。所提出的模型是一个说话人日记系统和混合自动语音识别(ASR)系统的组合。ASR系统中的说话人条件声学模型(SCAM)由一系列嵌入层组成,这些嵌入层使用来自日记化系统的说话人活动输入来导出说话人特定嵌入。SCAM的输出是用于解码对话中每个扬声器的转录的扬声器特定的senone。在这项工作中,我们的实验与自动扬声器活动的决定使用端到端扬声器日记系统。还提出了一种联合学习方法,其中日志化模型和ASR声学模型被联合优化。实验是在Switchboard电话会话语料库中的混合通道双说话人录音上进行的。在这些实验中,我们表明,所提出的声学模型,结合扬声器活动决策和联合优化,显着提高了ASR系统与显式源过滤(相对改善12%的字错误率(WER)的基线系统)。

英文摘要:

In this paper, we propose a novel approach for the transcription of speech conversations with natural speaker overlap, from single channel speech recordings. The proposed model is a combination of a speaker diarization system and a hybrid automatic speech recognition (ASR) system. The speaker conditioned acoustic model (SCAM) in the ASR system consists of a series of embedding layers which use the speaker activity inputs from the diarization system to derive speaker specific embeddings. The output of the SCAM are speaker specific senones that are used for decoding the transcripts for each speaker in the conversation. In this work, we experiment with the automatic speaker activity decisions generated using an end-to-end speaker diarization system. A joint learning approach is also proposed where the diarization model and the ASR acoustic model are jointly optimized. The experiments are performed on the mixed-channel two speaker recordings from the Switchboard corpus of telephone conversations. In these experiments, we show that the proposed acoustic model, incorporating speaker activity decisions and joint optimization, improves significantly over the ASR system with explicit source filtering (relative improvements of 12% in word error rate (WER) over the baseline system).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑