We propose a new transformer-based image and video tokenizer with Binary Spherical Quantization (BSQ). BSQ projects the high-dimensional visual embedding to a lower-dimensional hypersphere and then applies binary quantization. BSQ is (1) parameter-efficient without an explicit codebook, (2) scalable to arbitrary token dimensions, and (3) compact: compressing visual data by up to 100$\times$ with minimal distortion. Our tokenizer uses a transformer encoder and decoder with simple block-wise causal masking to support variable-length videos as input. The resulting BSQ-ViT achieves state-of-the-art visual reconstruction quality on image and video reconstruction benchmarks with 2.4$\times$ throughput compared to the best prior methods. Furthermore, by learning an autoregressive prior for adaptive arithmetic coding, BSQ-ViT achieves comparable results on video compression with state-of-the-art video compression standards. BSQ-ViT also enables masked language models to achieve competitive image synthesis quality to GAN- and diffusion-based methods.
LGQ: Learnable Geometric Quantization for Image Tokenization
Luddy School of Informatics, Computing and Engineering, Indiana University Bloomington（印第安纳大学布卢明顿分校信息学、计算与工程学院）；Department of Computer Science, University of Illinois Springfield（伊利诺伊大学斯普林菲尔德分校计算机科学系）；College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar（哈马德·本·卡伊夫大学多哈校区科学与工程学院）
Phaedra: Learning High-Fidelity Discrete Tokenization for the Physical Science
ETH AI Center（苏黎世联邦理工学院人工智能中心）；IBM Research Europe（IBM欧洲研究院）；Seminar for Applied Mathematics, ETH Zurich（苏黎世联邦理工学院应用数学系）；Swiss Data Science Center, ETH Zurich（瑞士数据科学中心，苏黎世联邦理工学院）
$I^{2}$-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence（人机混合增强智能国家级重点实验室）；Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（人工智能与机器人学院，西安交通大学）
ArcVQ-VAE: A Spherical Vector Quantization Framework with ArcCosine Additive Margin
Department of Artificial Intelligence, Chung-Ang University, Seoul, Republic of Korea（韩国首尔 Chung-Ang 大学人工智能系）；SNUAILAB, Seoul, Republic of Korea（韩国首尔 SNUAILAB 实验室）
Shanghai Innovation Institute（上海创新研究院）；Westlake University（西湖大学）；Zhejiang University（浙江大学）；Fudan University（复旦大学）；JD.COM（京东公司）；University of Chinese Academy of Sciences（中国科学院大学）
Hashing methods, which encode high-dimensional images with compact discrete codes, have been widely applied to enhance large-scale image retrieval. In this paper, we put forward Deep Spherical Quantization (DSQ), a novel method to make deep convolutional neural networks generate supervised and compact binary codes for efficient image search. Our approach simultaneously learns a mapping that transforms the input images into a low-dimensional discriminative space, and quantizes the transformed data points using multi-codebook quantization. To eliminate the negative effect of norm variance on cod…
WeTok: Powerful Discrete Tokenization for High-Fidelity Visual Reconstruction
Shanghai Jiao Tong University（上海交通大学）；WeChat Vision, Tencent Inc.（腾讯微信视觉部门）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；Hong Kong University of Science and Technology（香港科学大学）；Zhejiang University（浙江大学）；Shanghai AI Laboratory（上海人工智能实验室）
What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenization
Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning, Technical University of Munich（慕尼黑工业大学慕尼黑机器学习中心）；Klinikum rechts der Isar, Technical University of Munich（慕尼黑工业大学伊萨尔河右岸医院）；ELLIS Institute Finland（芬兰ELLIS研究所）；Aalto University（阿尔托大学）；Imperial College London（伦敦帝国理工学院）
M2Tok: Multi-head Multi-codebook Discrete Action Tokenization for Vision-Language-Action Models
The Hong Kong Polytechnic University（香港理工大学）；Shanghai AI Laboratory（上海人工智能实验室）；The University of Hong Kong（香港大学）；Shanghai Jiao Tong University（上海交通大学）；Hong Kong University of Science and Technology（香港科技大学）
HYDRA: Representation Harmonized Tokenization for Multimodal Generation and Understanding
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Tencent Hunyuan（腾讯文英）；Zhongguancun Academy（中关村学院）；Nanjing University（南京大学）；Peking University（北京大学）
Distributional Matching for Vector Quantization: A Unified Theoretical and Empirical Framework
University of Toronto（多伦多大学）；The Hong Kong University of Science and Technology（香港科技大学）；Boston College（波士顿学院）；Lehigh University（里海大学）；Southern University of Science and Technology（南方科技大学）
