语音分离引导的会议在线说话人日志
Online speaker diarization of meetings guided by speech separation
AI总结：
针对重叠语音干扰及语音分离模型难适配真实会议数据的问题，提出语音分离引导的在线说话人日志方案，经AMI语料微调后在AMI headset mix上达SOTA，重叠语音段表现尤佳。
摘要：
重叠语音对说话人日志系统而言是出了名的难题。因此，近期有研究提出利用语音分离来提升这类系统的性能。尽管前景可观，但语音分离模型在真实数据上表现不佳，因为它们是在说话人数量固定的模拟混合语音上训练的。本研究提出了一种新型语音分离引导的日志方案，适用于说话人数量可变的长会议录音的在线说话人日志任务，这类数据可参考AMI语料库（AMI corpus）。我们将ConvTasNet和DPRNN作为分离网络的备选方案，设置2个或3个输出源。为得到说话人日志结果，我们对每个估计出的源执行语音活动检测。最终模型先使用AMI数据让分离模块适配真实数据，再进行端到端微调。该系统基于短音频段运行，通过说话人嵌入和增量聚类拼接局部预测结果来完成推理。实验结果表明，在不使用先验信息、采用完整评估规则（无collar修正且包含重叠语音）的条件下，我们的系统刷新了AMI headset mix数据集上的当前最优性能。最后我们还证明，该系统在重叠语音片段上的表现尤为突出。
英文摘要：
Overlapped speech is notoriously problematic for speaker diarization systems. Consequently, the use of speech separation has recently been proposed to improve their performance. Although promising, speech separation models struggle with realistic data because they are trained on simulated mixtures with a fixed number of speakers. In this work, we introduce a new speech separation-guided diarization scheme suitable for the online speaker diarization of long meeting recordings with a variable number of speakers, as present in the AMI corpus. We envisage ConvTasNet and DPRNN as alternatives for the separation networks, with two or three output sources. To obtain the speaker diarization result, voice activity detection is applied on each estimated source. The final model is fine-tuned end-to-end, after first adapting the separation to real data using AMI. The system operates on short segments, and inference is performed by stitching the local predictions using speaker embeddings and incremental clustering. The results show that our system improves the state-of-the-art on the AMI headset mix, using no oracle information and under full evaluation (no collar and including overlapped speech). Finally, we show the strength of our system particularly on overlapped speech sections.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
