CodecSlime: 通过动态帧率压缩神经语音编解码器中的时间冗余
CodecSlime: Temporal Redundancy Compression of Neural Speech Codec via Dynamic Frame Rate
AI总结：
CodecSlime通过动态帧率压缩神经语音编解码器中的时间冗余，显著降低重建WER并实现比特率与质量的灵活权衡。
摘要：
神经语音编解码器已被广泛用于音频压缩和各种下游任务。当前主流编解码器为固定帧率（FFR），每等长时间片段分配相同数量的token。然而，语音在时间信息密度上本质上是非均匀的。因此，许多token被浪费在稳态段如长元音和沉默上。为了解决这种不匹配，我们提出了CodecSlime，一种插件式方法，通过在神经语音编解码器上支持动态帧率（DFR）来压缩时间冗余，这是首次实现。我们的方法是无监督的且架构无关的，结合了两个关键创新，ScheDFR和Melt-and-Cool，分别用于适应推理和训练。当整合到典型的VQ-GAN编解码器主干并以40 Hz DFR（≈600 bps）运行时，CodecSlime的重建WER比具有相同模型架构和相似比特率的常规FFR基线减少了多达32%，而其他指标也具有竞争力。CodecSlime还允许在重建质量与比特率之间进行灵活的权衡：单个模型支持在多个帧率下进行推理，并在相应的帧率下一致优于FFR模型。音频样本可在https://acadarmeria.github.io/codecslime/上获取。
英文摘要：
Neural speech codecs have been widely used in audio compression and various downstream tasks. Current mainstream codecs are fixed-frame-rate (FFR), which allocate the same number of tokens to every equal-duration slice. However, speech is inherently non-uniform in temporal information density. As a result, many tokens are wasted on steady-state segments like long vowels and silences. To address this mismatch, we present CodecSlime, a plugin-style method for compressing temporal redundancy through supporting dynamic frame rate (DFR) on neural speech codecs for the first time. Our method is unsupervised and architecture-agnostic, combining two key innovations, ScheDFR and Melt-and-Cool, for adapting inference and training, respectively. When integrated into a typical VQ-GAN codec backbone and operating at 40 Hz DFR ($\approx$ 600 bps), the reconstruction WER of CodecSlime is reduced by up to 32% relative to conventional FFR baselines with the same model architecture and similar bitrates, while other metrics are also competitive. CodecSlime also enables flexible trade-offs between reconstruction quality and bitrate: a single model supports inference at multiple frame rates and consistently outperforms FFR models at the corresponding frame rates. Audio samples are available at https://acadarmeria.github.io/codecslime/.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
