APCodec：一种具有并行幅度和相位谱编解码的神经音频编解码器
APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding
AI总结：
APCodec是一种新型神经音频编解码器，通过并行处理幅度和相位谱实现高采样率低比特率编码，在6 kbps下质量优于现有方法且生成更快。
摘要：
本文提出了一种新型神经音频编解码器，名为APCodec，旨在实现高波形采样率和低比特率，无缝融合了参数编解码器和波形编解码器的优势。APCodec通过并行处理幅度谱和相位谱作为音频参数特征，彻底改变了音频编解码过程。它由编码器和解码器组成，以改进的ConvNeXt v2网络为主干，通过基于残差矢量量化（RVQ）机制的量化器连接。编码器并行压缩音频幅度和相位谱，将其融合为时间分辨率降低的连续潜在码。该码随后由量化器量化。最终，解码器并行重建音频幅度和相位谱，通过逆短时傅里叶变换获得解码波形。为确保解码音频保真度，训练APCodec时联合采用频谱级损失、量化损失和生成对抗网络（GAN）损失。为支持低延迟流式推理，APCodec采用前馈层和因果反卷积层，并引入知识蒸馏训练策略提升解码音频质量。实验结果表明，APCodec能以6 kbps比特率编码48 kHz音频，解码质量无显著下降。在相同比特率下，与Encodec、AudioDec和DAC等知名编解码器相比，APCodec展现出更优的解码音频质量和更快的生成速度。
英文摘要：
This paper introduces a novel neural audio codec targeting high waveform sampling rates and low bitrates named APCodec, which seamlessly integrates the strengths of parametric codecs and waveform codecs. The APCodec revolutionizes the process of audio encoding and decoding by concurrently handling the amplitude and phase spectra as audio parametric characteristics like parametric codecs. It is composed of an encoder and a decoder with the modified ConvNeXt v2 network as the backbone, connected by a quantizer based on the residual vector quantization (RVQ) mechanism. The encoder compresses the audio amplitude and phase spectra in parallel, amalgamating them into a continuous latent code at a reduced temporal resolution. This code is subsequently quantized by the quantizer. Ultimately, the decoder reconstructs the audio amplitude and phase spectra in parallel, and the decoded waveform is obtained by inverse short-time Fourier transform. To ensure the fidelity of decoded audio like waveform codecs, spectral-level loss, quantization loss, and generative adversarial network (GAN) based loss are collectively employed for training the APCodec. To support low-latency streamable inference, we employ feed-forward layers and causal deconvolutional layers in APCodec, incorporating a knowledge distillation training strategy to enhance the quality of decoded audio. Experimental results confirm that our proposed APCodec can encode 48 kHz audio at bitrate of just 6 kbps, with no significant degradation in the quality of the decoded audio. At the same bitrate, our proposed APCodec also demonstrates superior decoded audio quality and faster generation speed compared to well-known codecs, such as Encodec, AudioDec and DAC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
