通过多尺度编码优化低比特率神经语音编解码器
Optimizing Neural Speech Codec for Low-Bitrate Compression via Multi-Scale Encoding
AI总结：
本文提出多尺度神经语音编解码器MsCodec，通过为不同信息密度的语音特征分配不同时间尺度的离散编码并引入互信息损失，解决了RVQ统一时间尺度导致的冗余编码问题，显著提升了低比特率下的语音压缩性能。
摘要：
神经语音编解码器已经展示了它们通过将高质量语音和音频转换为离散令牌表示来压缩这些信号的能力。大多数现有方法利用残差向量量化（RVQ）将语音编码为具有统一时间尺度的多层离散编码。然而，这种策略忽视了不同语音特征之间信息密度的差异，导致对稀疏信息的冗余编码，从而限制了这些方法在低比特率下的性能。本文提出了MsCodec，一种新颖的多尺度神经语音编解码器，它将语音编码为多层离散编码，每一层对应不同的时间尺度。这促使模型根据语音特征不同的信息密度对其进行解耦，从而提升语音压缩的性能。此外，我们引入互信息损失来增强不同层之间语音编码的多样性。实验结果表明，我们提出的方法显著提高了低比特率下的编解码器性能。
英文摘要：
Neural speech codecs have demonstrated their ability to compress high-quality speech and audio by converting them into discrete token representations. Most existing methods utilize Residual Vector Quantization (RVQ) to encode speech into multiple layers of discrete codes with uniform time scales. However, this strategy overlooks the differences in information density across various speech features, leading to redundant encoding of sparse information, which limits the performance of these methods at low bitrate. This paper proposes MsCodec, a novel multi-scale neural speech codec that encodes speech into multiple layers of discrete codes, each corresponding to a different time scale. This encourages the model to decouple speech features according to their diverse information densities, consequently enhancing the performance of speech compression. Furthermore, we incorporate mutual information loss to augment the diversity among speech codes across different layers. Experimental results indicate that our proposed method significantly improves codec performance at low bitrate.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
