arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2106.02896eess.AS
收藏

人类听力和实时字幕:用于语音增强的多任务训练

Human Listening and Live Captioning: Multi-Task Training for Speech Enhancement

Sefik Emre Eskimez, Xiaofei Wang, Min Tang, Hemin Yang, Zirun Zhu, Zhuo Chen, Huaming Wang, Takuya Yoshioka

摘要:

随着在线会议的激增,在各种噪声条件下提供高质量的语音音频和实时字幕变得比以往任何时候都更加重要。然而,大多数单声道语音增强(SE)模型引入处理伪像,从而降低下游任务(包括自动语音识别(ASR))的性能。本文提出了一个多任务训练框架,使SE模型无害的ASR。由于大多数ASR训练样本没有相应的干净信号参考,我们交替执行两个模型更新步骤,称为SE步骤和ASR步骤。SE步骤使用干净和有噪声的信号对以及基于信号的损失函数。ASR步骤将预训练的ASR模型应用于利用SE模型增强的训练信号。计算ASR输出和参考瞬态之间的交叉熵损失以更新SE模型参数。使用在75,000小时数据上训练的ASR模型进行实际大规模设置的实验结果表明,所提出的框架将SE输出的单词错误率提高了11.82%,而SE质量几乎没有影响。还通过改变ASR模型、用于ASR步骤的数据以及两个更新步骤的时间表来进行性能分析。

英文摘要:

With the surge of online meetings, it has become more critical than ever to provide high-quality speech audio and live captioning under various noise conditions. However, most monaural speech enhancement (SE) models introduce processing artifacts and thus degrade the performance of downstream tasks, including automatic speech recognition (ASR). This paper proposes a multi-task training framework to make the SE models unharmful to ASR. Because most ASR training samples do not have corresponding clean signal references, we alternately perform two model update steps called SE-step and ASR-step. The SE-step uses clean and noisy signal pairs and a signal-based loss function. The ASR-step applies a pre-trained ASR model to training signals enhanced with the SE model. A cross-entropy loss between the ASR output and reference transcriptions is calculated to update the SE model parameters. Experimental results with realistic large-scale settings using ASR models trained on 75,000-hour data show that the proposed framework improves the word error rate for the SE output by 11.82% with little compromise in the SE quality. Performance analysis is also carried out by changing the ASR model, the data used for the ASR-step, and the schedule of the two update steps.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑