ESC：基于跨尺度残差向量量化Transformer的高效语音编码
ESC: Efficient Speech Coding with Cross-Scale Residual Vector Quantized Transformers
AI总结：
本文提出高效语音编解码器ESC，通过跨尺度残差向量量化与镜像分层窗口Transformer实现从粗到细的逐步解码，以更低模型复杂度取得高保真语音重建效果。
摘要：
神经语音编解码器旨在以低延迟方式将输入信号压缩为最少比特，同时保持内容质量。然而，现有的神经编解码器常常以模型复杂度换取重建性能。这些编解码器主要使用卷积块进行特征变换，而卷积块本质上不适合捕捉语音信号中的局部冗余。为了弥补这一不足，它们需要对抗性判别器或大量模型参数来提升音频质量。针对这些挑战，我们提出了高效语音编解码器（Efficient Speech Codec，ESC），这是一种基于跨尺度残差向量量化方案和Transformer的轻量级、参数高效的语音编解码器。我们的模型采用镜像分层窗口Transformer块，并从粗到细的特征表示进行逐步解码。为了提高比特率效率，我们提出了一种新颖的向量量化技术组合以及预训练范式。大量实验表明，ESC能够以显著更低的模型复杂度实现高保真语音重建，使其成为现有卷积音频编解码器的一种有前景的替代方案。
英文摘要：
Neural speech codecs aim to compress input signals into minimal bits while maintaining content quality in a low-latency manner. However, existing neural codecs often trade model complexity for reconstruction performance. These codecs primarily use convolutional blocks for feature transformation, which are not inherently suited for capturing the local redundancies in speech signals. To compensate, they require either adversarial discriminators or a large number of model parameters to enhance audio quality. In response to these challenges, we introduce the Efficient Speech Codec (ESC), a lightweight, parameter-efficient speech codec based on a cross-scale residual vector quantization scheme and transformers. Our model employs mirrored hierarchical window transformer blocks and performs step-wise decoding from coarse-to-fine feature representations. To enhance bitrate efficiency, we propose a novel combination of vector quantization techniques along with a pre-training paradigm. Extensive experiments demonstrate that ESC can achieve high-fidelity speech reconstruction with significantly lower model complexity, making it a promising alternative to existing convolutional audio codecs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
