arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2204.03232eess.AScs.AIeess.SP
收藏

利用真实会话数据实现多通道连续语音分离

Leveraging Real Conversational Data for Multi-Channel Continuous Speech Separation

Microsoft Cloud+AI(微软云与人工智能部门)

Xiaofei Wang, Dongmei Wang, Naoyuki Kanda, Sefik Emre Eskimez, Takuya Yoshioka

摘要:

现有的多通道连续语音分离(CSS)模型严重依赖于监督数据-要么是导致训练和真实数据测试之间的数据不匹配的模拟数据,要么是难以获得的真实转录重叠数据,阻碍了会话/会议转录任务的进一步改进。在本文中,我们为CSS模型提出了一个三阶段的训练方案,该方案可以利用监督数据和超大规模的无监督真实世界对话数据。该方案包括两种传统的训练方法-使用模拟数据的预训练和使用转录数据的基于ASR损失的训练-以及两者之间的新型连续半监督训练,其中CSS模型通过使用基于师生学习框架的真实数据进一步训练。我们将此方案应用于阵列几何无关的CSS模型,该模型可以使用从任何麦克风阵列收集的多通道数据。在Microsoft内部会议数据和AMI会议语料库上进行了大规模的会议转写实验。已经观察到每个训练阶段的稳定改进,显示了所提出的方法的效果,该方法能够利用真实的会话数据进行CSS模型训练。

英文摘要:

Existing multi-channel continuous speech separation (CSS) models are heavily dependent on supervised data - either simulated data which causes data mismatch between the training and real-data testing, or the real transcribed overlapping data, which is difficult to be acquired, hindering further improvements in the conversational/meeting transcription tasks. In this paper, we propose a three-stage training scheme for the CSS model that can leverage both supervised data and extra large-scale unsupervised real-world conversational data. The scheme consists of two conventional training approaches -- pre-training using simulated data and ASR-loss-based training using transcribed data -- and a novel continuous semi-supervised training between the two, in which the CSS model is further trained by using real data based on the teacher-student learning framework. We apply this scheme to an array-geometry-agnostic CSS model, which can use the multi-channel data collected from any microphone array. Large-scale meeting transcription experiments are carried out on both Microsoft internal meeting data and the AMI meeting corpus. The steady improvement by each training stage has been observed, showing the effect of the proposed method that enables leveraging real conversational data for CSS model training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑