arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.17133cs.LGcs.AI
收藏

VP-VAE:通过自适应向量扰动重新思考向量量化

VP-VAE: Rethinking Vector Quantization via Adaptive Vector Perturbation

Xi'an Jiaotong University, China(西安交通大学)

Linwei Zhai, Han Ding, Mingzhi Lin, Cui Zhao, Fei Wang, Ge Wang, Wang Zhi, Wei Xi

AI总结:

VP-VAE通过自适应向量扰动方法解耦表示学习与离散化,提升生成模型的稳定性与重建保真度。

摘要:

向量量化变分自编码器(VQ-VAEs)是现代生成建模的基础,但它们常常因表示学习与离散代码本优化的内在耦合而遭受训练不稳定和

英文摘要:

Vector Quantized Variational Autoencoders (VQ-VAEs) are fundamental to modern generative modeling, yet they often suffer from training instability and "codebook collapse" due to the inherent coupling of representation learning and discrete codebook optimization. In this paper, we propose VP-VAE (Vector Perturbation VAE), a novel paradigm that decouples representation learning from discretization by eliminating the need for an explicit codebook during training. Our key insight is that, from the neural network's viewpoint, performing quantization primarily manifests as injecting a structured perturbation in latent space. Accordingly, VP-VAE replaces the non-differentiable quantizer with distribution-consistent and scale-adaptive latent perturbations generated via Metropolis--Hastings sampling. This design enables stable training without a codebook while making the model robust to inference-time quantization error. Moreover, under the assumption of approximately uniform latent variables, we derive FSP (Finite Scalar Perturbation), a lightweight variant of VP-VAE that provides a unified theoretical explanation and a practical improvement for FSQ-style fixed quantizers. Extensive experiments on image and audio benchmarks demonstrate that VP-VAE and FSP improve reconstruction fidelity and achieve substantially more balanced token usage, while avoiding the instability inherent to coupled codebook training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑