arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2109.12591cs.SDeess.AS
收藏

基于Cycle-in-Cycle GAN的非并行语音增强联合幅度估计和相位恢复

Joint magnitude estimation and phase recovery using Cycle-in-Cycle GAN for non-parallel speech enhancement

Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);Bytedance(字节跳动)

Guochen Yu, Andong Li, Yutian Wang, Yinuo Guo, Hui Wang, Chengshi Zheng

摘要:

由于在许多实际应用中缺乏足够的成对噪声-干净语音语料,非并行训练是基于DNN的语音增强方法的一个很有前途的任务。然而,由于输入语音和目标语音之间存在严重的失配,以往的研究往往只关注幅度谱估计,而不改变相位,导致低信噪比条件下语音质量下降。为了解决这个问题,我们将困难的目标w.r.t.将原始谱优化分解为谱幅度和谱相位,提出了一种新的循环中循环生成对抗网络(CinCGAN),用于在不成对数据下逐级联合估计谱幅度和相位信息。在第一阶段,我们预训练幅度CycleGAN来粗略估计干净语音的频谱幅度。在第二阶段,我们将预训练的CycleGAN与复值CycleGAN结合起来作为循环中循环结构,以同时恢复相位信息并细化整体频谱。实验结果表明,所提出的方法显着优于以前的基线下非并行训练。对标准配对数据训练模型的评价也表明,CinCGAN在降低背景噪声和语音失真方面取得了显著的效果。

英文摘要:

For the lack of adequate paired noisy-clean speech corpus in many real scenarios, non-parallel training is a promising task for DNN-based speech enhancement methods. However, because of the severe mismatch between input and target speeches, many previous studies only focus on the magnitude spectrum estimation and remain the phase unaltered, resulting in the degraded speech quality under low signal-to-noise ratio conditions. To tackle this problem, we decouple the difficult target w.r.t. original spectrum optimization into spectral magnitude and phase, and a novel Cycle-in-Cycle generative adversarial network (dubbed CinCGAN) is proposed to jointly estimate the spectral magnitude and phase information stage by stage under unpaired data. In the first stage, we pretrain a magnitude CycleGAN to coarsely estimate the spectral magnitude of clean speech. In the second stage, we incorporate the pretrained CycleGAN with a complex-valued CycleGAN as a cycle-in-cycle structure to simultaneously recover phase information and refine the overall spectrum. Experimental results demonstrate that the proposed approach significantly outperforms previous baselines under non-parallel training. The evaluation on training the models with standard paired data also shows that CinCGAN achieves remarkable performance especially in reducing background noise and speech distortion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑