The Hong Kong University of Science and Technology(香港科学与技术大学);Horizon Robotics(Horizon机器人);Nanjing University(南京大学);Hong Kong University(香港大学);The Chinese University of Hong Kong(香港中文大学)
Recent advances in visual generation have emphasized the importance of Latent Generative Models (LGMs), which critically depend on effective visual tokenizers to bridge pixels and semantic representations. However, tokenizers constructed on pre-trained vision foundation models (VFMs) often struggle to balance semantic richness and reconstruction fidelity in high-dimensional latent spaces. In this paper, we introduce DINO-Tok, a visual tokenizer built upon a frozen DINO encoder that supports both continuous autoencoding (DINO-Tok-AE) and discrete vector-quantization (DINO-Tok-VQ). By unifying hierarchical representations from both shallow fine-grained features and deep global semantics into an information-complete latent space, DINO-Tok preserves texture details while maintaining \textit{semantic consistency} for generation. We further investigate VQ in frozen semantic feature spaces of high dimensionality, where information dilution and codebook collapse frequently arise. To address this issue, we propose Dominant-Subspace Quantization (DSQ), which leverages a global PCA analysis to select principal components while suppressing noisy dimensions, thereby stabilizing codebook optimization and improving reconstruction and generation quality. On ImageNet 256x256, DINO-Tok achieves strong reconstruction performance, achieving 0.28 rFID for continuous autoencoding and 1.10 rFID for discrete VQ, as well as strong few-step generation performance 1.82 gFID for diffusion and 2.44 gFID for autoregressive generation. These results demonstrate that pre-trained VFMs such as DINO can be directly adapted into high-fidelity, semantically aligned visual tokenizers for next-generation latent generative models. Code will be publicly available at https://github.com/MKJia/DINO-Tok.
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Dalian University of Technology(大连理工大学);Tsinghua University(清华大学)
针对视觉分词器与下游任务优化脱节导致的表示瓶颈,提出 ETT 端到端联合调优方法,在多模态理解与生成任务上较冻结基线提升 2-6%。
WinTok: A Win-Win Hybrid Tokenizer via Decomposing Visual Understanding and Generation with Transferable Tokens
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);WeChat Vision, Tencent Inc.(腾讯微信视觉部);Shanghai Jiao Tong University(上海交通大学)
RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身人工智能研究所);Robbyant, Ant Group(蚂蚁集团 Robbyant);Hongkong University of Science and Technology(香港科技大学)
VFM-VAE: Vision Foundation Models Can Be Good Tokenizers for Latent Diffusion Models
State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(人机混合增强智能国家重点实验室,人工智能与机器人研究院,西安交通大学);Microsoft Research Asia(微软亚洲研究院)
What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization
Technical University of Munich(慕尼黑工业大学);Munich Center for Machine Learning, Technical University of Munich(慕尼黑工业大学慕尼黑机器学习中心);Klinikum rechts der Isar, Technical University of Munich(慕尼黑工业大学伊萨尔河右岸医院);ELLIS Institute Finland(芬兰ELLIS研究所);Aalto University(阿尔托大学);Imperial College London(伦敦帝国理工学院)