MBCodec：面向高保真音频压缩的彻底解耦
MBCodec:Thorough disentangle for high-fidelity audio compression
AI总结：
本文提出MBCodec，一种基于残差向量量化的多码本音频编解码器，通过自监督语义标记、子带特征和自适应训练策略彻底解耦语义与声学信息，实现24 kHz音频170倍压缩至2.2 kbps的近无损重建。
摘要：
文本到语音（TTS）中的高保真神经音频编解码器旨在将语音信号压缩为离散表示，以实现忠实重建。然而，先前的方法在有效解耦令牌中的声学信息和语义信息方面面临挑战，导致合成语音缺乏细粒度细节。在本研究中，我们提出了MBCodec，这是一种基于残差向量量化（RVQ）的新型多码本音频编解码器，可学习分层结构化表示。MBCodec利用自监督语义标记化和来自原始信号的音频子带特征，构建功能解耦的潜在空间。为了促进编解码器嵌入空间各层的全面学习，我们引入自适应丢弃深度以对不同层的码本进行差异化训练，并在训练过程中采用多通道伪正交镜像滤波器（PQMF）。通过彻底解耦语义特征和声学特征，我们的方法不仅实现了近无损的语音重建，还实现了24 kHz音频的显著170倍压缩，从而获得仅2.2 kbps的低比特率。实验评估证实，其在所有评估中均一致且显著地优于基线。
英文摘要：
High-fidelity neural audio codecs in Text-to-speech (TTS) aim to compress speech signals into discrete representations for faithful reconstruction. However, prior approaches faced challenges in effectively disentangling acoustic and semantic information within tokens, leading to a lack of fine-grained details in synthesized speech. In this study, we propose MBCodec, a novel multi-codebook audio codec based on Residual Vector Quantization (RVQ) that learns a hierarchically structured representation. MBCodec leverages self-supervised semantic tokenization and audio subband features from the raw signals to construct a functionally-disentangled latent space. In order to encourage comprehensive learning across various layers of the codec embedding space, we introduce adaptive dropout depths to differentially train codebooks across layers, and employ a multi-channel pseudo-quadrature mirror filter (PQMF) during training. By thoroughly decoupling semantic and acoustic features, our method not only achieves near-lossless speech reconstruction but also enables a remarkable 170x compression of 24 kHz audio, resulting in a low bit rate of just 2.2 kbps. Experimental evaluations confirm its consistent and substantial outperformance of baselines across all evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
