HiFTNet：一种具有谐波加噪声滤波器和逆短时傅里叶变换的快速高质量神经声码器
HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform
AI总结：
本文提出HiFTNet，通过在iSTFTNet中引入谐波加噪声源滤波器并利用预训练F0估计网络，在保持高质量语音合成的同时显著降低计算成本和参数规模，速度达BigVGAN四倍且参数仅为其1/6。
摘要：
语音合成的最新进展利用基于GAN的网络（如HiFi-GAN和BigVGAN）从梅尔频谱图生成高保真波形。然而，这些网络计算开销大且参数众多。iSTFTNet通过将逆短时傅里叶变换（iSTFT）集成到网络中来解决这些限制，同时实现了速度和参数效率。在本文中，我们提出了iSTFTNet的一个扩展，称为HiFTNet，它在时频域中引入了一个谐波加噪声源滤波器，该滤波器使用由预训练F0估计网络推断出的基频（F0）产生的正弦源，以实现快速推理速度。在LJSpeech上的主观评估表明，我们的模型显著优于iSTFTNet和HiFi-GAN，达到了真实录音级别的性能。HiFTNet在LibriTTS上对未见过的说话人也优于BigVGAN-base，并且在与BigVGAN性能相当的同时，速度快四倍，参数仅为BigVGAN的1/6。我们的工作为高效、高质量的神经声码设立了新的基准，为需要高质量语音合成的实时应用铺平了道路。
英文摘要：
Recent advancements in speech synthesis have leveraged GAN-based networks like HiFi-GAN and BigVGAN to produce high-fidelity waveforms from mel-spectrograms. However, these networks are computationally expensive and parameter-heavy. iSTFTNet addresses these limitations by integrating inverse short-time Fourier transform (iSTFT) into the network, achieving both speed and parameter efficiency. In this paper, we introduce an extension to iSTFTNet, termed HiFTNet, which incorporates a harmonic-plus-noise source filter in the time-frequency domain that uses a sinusoidal source from the fundamental frequency (F0) inferred via a pre-trained F0 estimation network for fast inference speed. Subjective evaluations on LJSpeech show that our model significantly outperforms both iSTFTNet and HiFi-GAN, achieving ground-truth-level performance. HiFTNet also outperforms BigVGAN-base on LibriTTS for unseen speakers and achieves comparable performance to BigVGAN while being four times faster with only $1/6$ of the parameters. Our work sets a new benchmark for efficient, high-quality neural vocoding, paving the way for real-time applications that demand high quality speech synthesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
