arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2511.20565cs.CV
收藏

DINO-Tok:为视觉分词器适应DINO

DINO-Tok: Adapting DINO for Visual Tokenizers

The Hong Kong University of Science and Technology(香港科学与技术大学);Horizon Robotics(Horizon机器人);Nanjing University(南京大学);Hong Kong University(香港大学);The Chinese University of Hong Kong(香港中文大学)
Mingkai Jia, Mingxiao Li, Zhijian Shu, Anlin Zheng, Liaoyuan Fan, Jiaxin Guo, Tianxing Shi, Dongyue Lu, Zeming Li, Xiaoyang Guo, Xiaojuan Qi, Xiao-Xiao Long, Qi… 展开作者

Mingkai Jia, Mingxiao Li, Zhijian Shu, Anlin Zheng, Liaoyuan Fan, Jiaxin Guo, Tianxing Shi, Dongyue Lu, Zeming Li, Xiaoyang Guo, Xiaojuan Qi, Xiao-Xiao Long, Qian Zhang, Ping Tan, Wei Yin

AI总结:

本文提出DINO-Tok,一种基于冻结DINO编码器的视觉分词器,结合连续自编码和离散向量量化方法,提升高维潜在空间中的语义一致性和生成质量。

摘要:

近期视觉生成的进展强调了潜在生成模型(LGMs)的重要性,其关键依赖于有效的视觉分词器以连接像素和语义表示。然而,基于预训练视觉基础模型(VFMs)构建的分词器在高维潜在空间中难以平衡语义丰富性和重建保真度。本文引入DINO-Tok,一种基于冻结DINO编码器的视觉分词器,支持连续自编码(DINO-Tok-AE)和离散向量量化(DINO-Tok-VQ)。通过统一浅层细粒度特征和深层全局语义的分层表示,DINO-Tok在保持纹理细节的同时维持语义一致性。进一步研究了高维冻结语义特征空间中的向量量化,其中信息稀释和代码书崩溃频繁出现。为此,我们提出主导子空间量化(DSQ),利用全局PCA分析选择主成分并抑制噪声维度,从而稳定代码书优化并提高重建和生成质量。在ImageNet 256x256上,DINO-Tok在连续自编码中实现0.28 rFID,在离散VQ中实现1.10 rFID,且在扩散和自回归生成中分别达到1.82 gFID和2.44 gFID。这些结果表明,预训练的VFMs如DINO可以直接适应为高保真、语义对齐的视觉分词器,用于下一代潜在生成模型。代码将在https://github.com/MKJia/DINO-Tok公开发布。

英文摘要:

Recent advances in visual generation have emphasized the importance of Latent Generative Models (LGMs), which critically depend on effective visual tokenizers to bridge pixels and semantic representations. However, tokenizers constructed on pre-trained vision foundation models (VFMs) often struggle to balance semantic richness and reconstruction fidelity in high-dimensional latent spaces. In this paper, we introduce DINO-Tok, a visual tokenizer built upon a frozen DINO encoder that supports both continuous autoencoding (DINO-Tok-AE) and discrete vector-quantization (DINO-Tok-VQ). By unifying hierarchical representations from both shallow fine-grained features and deep global semantics into an information-complete latent space, DINO-Tok preserves texture details while maintaining \textit{semantic consistency} for generation. We further investigate VQ in frozen semantic feature spaces of high dimensionality, where information dilution and codebook collapse frequently arise. To address this issue, we propose Dominant-Subspace Quantization (DSQ), which leverages a global PCA analysis to select principal components while suppressing noisy dimensions, thereby stabilizing codebook optimization and improving reconstruction and generation quality. On ImageNet 256x256, DINO-Tok achieves strong reconstruction performance, achieving 0.28 rFID for continuous autoencoding and 1.10 rFID for discrete VQ, as well as strong few-step generation performance 1.82 gFID for diffusion and 2.44 gFID for autoregressive generation. These results demonstrate that pre-trained VFMs such as DINO can be directly adapted into high-fidelity, semantically aligned visual tokenizers for next-generation latent generative models. Code will be publicly available at https://github.com/MKJia/DINO-Tok.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑