NDVQ：基于正态分布向量量化的鲁棒神经音频编解码器
NDVQ: Robust Neural Audio Codec with Normal Distribution-Based Vector Quantization
AI总结：
针对极低带宽下VQ码本对噪声敏感导致音频质量下降的问题，提出基于正态分布的向量量化方法NDVQ，通过学习方差引入码间间隔，提升音频压缩与零样本TTS性能。
摘要：
基于向量量化（VQ）的离散音频编解码模型在音频压缩和自回归音频生成方面取得了巨大成功。然而，现有模型在感知质量和信号失真方面面临重大挑战，尤其是在极低带宽条件下运行时，这源于VQ码本对噪声的敏感性。这种退化对若干下游任务（如基于编解码器的语音合成）构成了重大挑战。为解决此问题，我们提出了一种新颖的VQ方法——基于正态分布的向量量化（NDVQ），通过学习方差在VQ码之间引入显式间隔。具体而言，我们的方法将波形映射到潜在空间，并通过选择最可能的正态分布对其进行量化，每个码本条目代表一个由其均值和方差定义的独特正态分布。使用这些基于分布的VQ编解码码，解码器重建输入波形。NDVQ在重建损失和判别损失之外，还使用额外的分布相关损失进行训练。实验表明，NDVQ在音频质量和零样本TTS方面优于现有音频压缩基线（如EnCodec），尤其是在极低带宽场景下。
英文摘要：
Built upon vector quantization (VQ), discrete audio codec models have achieved great success in audio compression and auto-regressive audio generation. However, existing models face substantial challenges in perceptual quality and signal distortion, especially when operating in extremely low bandwidth, rooted in the sensitivity of the VQ codebook to noise. This degradation poses significant challenges for several downstream tasks, such as codec-based speech synthesis. To address this issue, we propose a novel VQ method, Normal Distribution-based Vector Quantization (NDVQ), by introducing an explicit margin between the VQ codes via learning a variance. Specifically, our approach involves mapping the waveform to a latent space and quantizing it by selecting the most likely normal distribution, with each codebook entry representing a unique normal distribution defined by its mean and variance. Using these distribution-based VQ codec codes, a decoder reconstructs the input waveform. NDVQ is trained with additional distribution-related losses, alongside reconstruction and discrimination losses. Experiments demonstrate that NDVQ outperforms existing audio compression baselines, such as EnCodec, in terms of audio quality and zero-shot TTS, particularly in very low bandwidth scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
