AffectCodec: 具有块对角残差FSQ的情感保持神经语音编解码器
AffectCodec: Emotion-Preserving Neural Speech Codec with Block-Diagonal Residual FSQ
AI总结：
针对神经语音编解码器在量化过程中丢弃情感线索的问题，提出基于块对角残差有限标量量化（BD-RFSQ）的AffectCodec，通过结构约束的量化实现情感保持，在低比特率下显著提升情感保留度。
摘要：
神经语音编解码器已成为原始音频与语音语言模型之间的离散接口，但它们仍主要针对声学重建保真度进行优化，这导致情感相关线索在量化过程中容易被丢弃，限制了下游模型的情感能力。我们将这种退化追溯到两种机制：有限比特率下由重建驱动的比特分配，以及基于拼接的编解码器中的跨流泄漏，其中声学梯度可能覆盖名义上保留情感的维度。我们提出AffectCodec，一种基于块对角残差有限标量量化（BD-RFSQ）的情感保持神经语音编解码器。通过在情感和声学子空间上施加块对角输入和输出投影，BD-RFSQ将比特分配从隐式和损失驱动转变为显式和结构保证，同时为下游语音语言模型保留平坦的令牌接口。AffectCodec进一步将这种结构约束的量化器与多粒度情感条件和多速率训练相结合，在低比特率下实现鲁棒的情感保持。在多个情感语音基准上的实验表明，AffectCodec显著改善了情感保持，尤其是在低比特率情况下，同时保持了具有竞争力的声学质量和可懂度。这些结果表明，结构保护的量化是保留情感相关信息的有效原则，并可能为属性感知的神经语音压缩提供一条通用途径。
英文摘要：
Neural speech codecs have become the discrete interface between raw audio and speech language models, yet they remain optimized primarily for acoustic reconstruction fidelity, which leaves emotion-relevant cues vulnerable to being discarded during quantization, limiting the affective capacity of downstream models. We trace this degradation to two mechanisms: reconstruction-driven bit allocation under limited bitrate and cross-stream leakage in concatenation-based codecs, where acoustic gradients can overwrite nominally emotion-reserved dimensions. We propose AffectCodec, an emotion-preserving neural speech codec built on Block-Diagonal Residual Finite Scalar Quantization (BD-RFSQ). By imposing block-diagonal input and output projections over emotion and acoustic subspaces, BD-RFSQ transforms bit allocation from implicit and loss-driven to explicit and structurally guaranteed, while still preserving a flat token interface for downstream speech language models. AffectCodec further combines this structurally constrained quantizer with multi-granularity emotion conditioning and multi-rate training, enabling robust affect preservation at low bitrates. Experiments across multiple emotional speech benchmarks show that AffectCodec substantially improves emotion preservation, especially in the low-bitrate regime, while maintaining competitive acoustic quality and intelligibility. These results suggest that structurally protected quantization is an effective principle for preserving emotion-relevant information and may provide a general route toward attribute-aware neural speech compression.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
