自引导：通过解码器流形对齐增强神经编解码器
Self-Guidance: Enhancing Neural Codecs via Decoder Manifold Alignment
AI总结：
提出自引导方法，通过轻量特征映射损失对齐解码器内部流形，在不改变推理过程下提升VQ-VAE神经语音编解码器重建质量，实现低比特率SOTA性能并支持4倍码本缩减。
摘要：
基于向量量化VAE（VQ-VAE）的神经语音编解码器是语音大语言模型的核心音频分词器，但其重建保真度受限于量化误差。常见的修复方法是修改量化器或增加模型容量，但这会复杂化下游语言建模。我们的核心思想是，在处理量化标记及其原始连续嵌入时，使用轻量级特征映射损失对齐解码器的内部特征流形。这需要最小的训练开销，且无需改变推理过程。应用于XCodec2时，自引导改善了所有重建指标，实现了低比特率下的最先进性能。值得注意的是，它实现了4倍码本缩减而无保真度损失，下游TTS实验表明，通过简化标记建模空间，这显著改善了基于LLM的合成。多项统计观察和可视化证实了解码器中内部流形对齐的增强。大量实验证实了其在各种归纳偏置下的通用性。因此，自引导建立了一种高效、广泛适用的高保真神经音频编码方法。
英文摘要：
Neural speech codecs based on Vector-Quantized VAEs (VQ-VAEs) are core audio tokenizers for speech LLMs, yet their reconstruction fidelity is bottlenecked by quantization error. Modifying the quantizer or increasing model capacity are common fixes, but they complicate downstream language modeling. Our core idea is to align the decoder's internal feature manifolds when processing both the quantized tokens and their original continuous embeddings, using a lightweight feature-mapping loss. This requires minimal training overhead and no inference-time changes. Applied to XCodec2, self-guidance improves all reconstruction metrics, achieving state-of-the-art low-bitrate performance. Notably, it enables a 4x codebook reduction without fidelity loss, which downstream TTS experiments show significantly improves LLM-based synthesis by simplifying the token modeling space. Multiple statistical observations and visualizations corroborate the enhanced internal manifold alignment in the decoder. Extensive experiments confirm its generality across various inductive biases. Self-guidance thus establishes an efficient, broadly applicable method for high-fidelity neural audio coding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
