语谱图块编解码器：用于神经语音编码的2D块量化VQ-VAE与HiFi-GAN
Spectrogram Patch Codec: A 2D Block-Quantized VQ-VAE and HiFi-GAN for Neural Speech Coding
AI总结：
本研究提出一种基于mel语谱图4×4块单阶段量化的神经语音编解码器，结合VQ-VAE后期对抗微调与HiFi-GAN声码器，在7.5kbits/s速率下达到与现有先进方案相当的语音质量，为低延迟编解码器提供开放基础。
摘要：
我们提出了一种神经语音编解码器，通过引入更简单的单阶段量化方法，挑战了对复杂残差矢量量化（RVQ）堆叠的需求。我们的方法直接在mel语谱图上运行，将其视为二维数据，将不重叠的4×4块量化到单个共享码本中。这种逐块设计简化了架构，支持低延迟流式传输，并生成离散潜在网格。为确保高保真合成，我们对VQ-VAE采用了后期对抗性微调，并在该编解码器的重建语谱图上从头训练了HiFi-GAN声码器。对于16kHz语音，我们的系统以约7.5kbits/s的速率运行，我们使用STOI、PESQ、MCD和ViSQOL等客观指标，将其与多种最先进的神经编解码器进行了对比评估。结果表明，我们这种简化的非残差架构实现了具有竞争力的感知质量和可懂度，验证了它可作为未来低延迟编解码器设计的有效且开放的基础。
英文摘要：
We present a neural speech codec that challenges the need for complex residual vector quantization (RVQ) stacks by introducing a simpler, single-stage quantization approach. Our method operates directly on the mel-spectrogram, treating it as a 2D data and quantizing non-overlapping 4x4 patches into a single, shared codebook. This patchwise design simplifies the architecture, enables low-latency streaming, and yields a discrete latent grid. To ensure high-fidelity synthesis, we employ a late-stage adversarial fine-tuning for the VQ-VAE and train a HiFi-GAN vocoder from scratch on the codec's reconstructed spectrograms. Operating at approximately 7.5 kbits/s for 16 kHz speech, our system was evaluated against several state-of-the-art neural codecs using objective metrics such as STOI, PESQ, MCD, and ViSQOL. The results demonstrate that our simplified, non-residual architecture achieves competitive perceptual quality and intelligibility, validating it as an effective and open foundation for future low-latency codec designs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
