arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.14941eess.AScs.AIcs.LGcs.SDeess.SP
收藏

Neural Vocoder is All You Need for Speech Super-resolution(神经声码器是语音超分辨率所需的全部)

Neural Vocoder is All You Need for Speech Super-resolution

University of Surrey(萨里大学);Queen Mary University of London(伦敦玛丽女王大学);ByteDance(字节跳动);The Ohio State University(俄亥俄州立大学)

Haohe Liu, Woosung Choi, Xubo Liu, Qiuqiang Kong, Qiao Tian, DeLiang Wang

AI总结:

针对现有语音超分辨率方法受固定上采样比等约束、泛化性差的问题,提出含三模块的NVSR方法,在VCTK基准上取得最优性能,验证了预训练声码器先验知识的关键作用。

摘要:

语音超分辨率(SR)是一项通过生成高频分量来提高语音采样率的任务。现有语音SR方法在受限实验设置下训练,例如固定上采样比。这些强约束可能导致在不匹配的真实场景中泛化能力较差。本文提出一种基于神经声码器的语音超分辨率方法(NVSR),可处理多种输入分辨率和上采样比。NVSR由梅尔带宽扩展模块、神经声码器模块和后处理模块组成。我们提出的系统在VCTK多说话人基准上取得了最优结果。在44.1 kHz目标分辨率下,NVSR在对数谱距离上分别优于WSRGlow和Nu-wave 8%和37%,并实现了显著更好的感知质量。我们还通过使用简单的复制填充方法进行梅尔带宽扩展,证明了预训练声码器中的先验知识对语音SR至关重要。样本可在https://haoheliu.github.io/nvsr查看。

英文摘要:

Speech super-resolution (SR) is a task to increase speech sampling rate by generating high-frequency components. Existing speech SR methods are trained in constrained experimental settings, such as a fixed upsampling ratio. These strong constraints can potentially lead to poor generalization ability in mismatched real-world cases. In this paper, we propose a neural vocoder based speech super-resolution method (NVSR) that can handle a variety of input resolution and upsampling ratios. NVSR consists of a mel-bandwidth extension module, a neural vocoder module, and a post-processing module. Our proposed system achieves state-of-the-art results on the VCTK multi-speaker benchmark. On 44.1 kHz target resolution, NVSR outperforms WSRGlow and Nu-wave by 8% and 37% respectively on log spectral distance and achieves a significantly better perceptual quality. We also demonstrate that prior knowledge in the pre-trained vocoder is crucial for speech SR by performing mel-bandwidth extension with a simple replication-padding method. Samples can be found in https://haoheliu.github.io/nvsr.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑