This paper presents a waveform modeling and generation method using hierarchical recurrent neural networks (HRNN) for speech bandwidth extension (BWE). Different from conventional BWE methods which predict spectral parameters for reconstructing wideband speech waveforms, this BWE method models and predicts waveform samples directly without using vocoders. Inspired by SampleRNN which is an unconditional neural audio generator, the HRNN model represents the distribution of each wideband or high-frequency waveform sample conditioned on the input narrowband waveform samples using a neural network composed of long short-term memory (LSTM) layers and feed-forward (FF) layers. The LSTM layers form a hierarchical structure and each layer operates at a specific temporal resolution to efficiently capture long-span dependencies between temporal sequences. Furthermore, additional conditions, such as the bottleneck (BN) features derived from narrowband speech using a deep neural network (DNN)-based state classifier, are employed as auxiliary input to further improve the quality of generated wideband speech. The experimental results of comparing several waveform modeling methods show that the HRNN-based method can achieve better speech quality and run-time efficiency than the dilated convolutional neural network (DCNN)-based method and the plain sample-level recurrent neural network (SRNN)-based method. Our proposed method also outperforms the conventional vocoder-based BWE method using LSTM-RNNs in terms of the subjective quality of the reconstructed wideband speech.
Qualcomm AI Research, Qualcomm Technologies, Inc.（高通技术公司高通人工智能研究院）；Qualcomm AI Research, Qualcomm Technologies Netherlands B.V.（高通技术荷兰私人有限公司高通人工智能研究院）；University of California San Diego（加州大学圣地亚哥分校）
A Waveform Representation Framework for High-quality Statistical Parametric Speech Synthesis
Northwestern Polytechnical University（西北工业大学）；Institute for Infocomm Research（资讯通信研究院）；Nanyang Technological University (NTU)（南洋理工大学）；Joint NTU-UBC Research Center of Excellence in Active Living for the Elderly（南洋理工大学-不列颠哥伦比亚大学活跃老年生活卓越联合研究中心）
Waveform to Single Sinusoid Regression to Estimate the F0 Contour from Noisy Speech Using Recurrent Deep Neural Networks
University of Eastern Finland（东芬兰大学）
基频（F0）表示语音中的音高，其确定语音的韵律特征，并且在用于语音分析和合成的各种任务中是需要的。尽管对这一主题进行了数十年的研究，但在意外噪声条件下的低信噪比（SNR）下的F0估计仍然很困难。这项工作提出了一种新的方法，噪声鲁棒F0估计使用的递归神经网络（RNN）训练的监督方式。最近的研究采用深度神经网络（DNN）进行F0跟踪，作为量化频率状态的逐帧分类任务，但我们提出了波形到正弦曲线回归，以实现噪声鲁棒性和准确估计，同时提高频率分辨率。在加性噪声污染的PTDB-TUG语料库（NOISEX-92）上的实验结果表明，与著名的噪声鲁棒F0跟踪器PEFAC相比，该方法在信噪比为-10 dB ~+10 dB时，粗基音误差（GPE）率和细基音误差（FPE）率均提高了35%以上。此外，所提出的方法在先前SNR范围内的FPE和GPE速率方面也优于最先进的基于DNN的方法超过15%。
Towards High-Quality and Efficient Speech Bandwidth Extension with Parallel Amplitude and Phase Prediction
National Engineering Research Center of Speech and Language Information Processing（国家语音与语言信息处理工程研究中心）；University of Science and Technology of China（中国科学技术大学）
