Unified multimodal large language models (MLLMs) aim to unify image understanding and image generation within a single framework, where a shared visual tokenizer serves as the sole interface that maps high-dimensional images into a limited token budget for downstream multimodal reasoning and synthesis. However, existing shared-token designs are largely architecture-driven and lack an explicit criterion for what information should be preserved to simultaneously support semantic abstraction and visual detail. In this paper, we adopt a capacity-constrained perspective, viewing the shared tokenizer as a compute-bounded learner whose finite representational budget should prioritize reusable structure over hard-to-exploit high-entropy variations and redundancy. Motivated by this view, we propose \textbf{\textit{InfoTok}}, an information-regularized tokenization mechanism grounded in the Information Bottleneck (IB) principle. InfoTok explicitly controls information flow from images to shared tokens to multimodal outputs by imposing mutual-information (MI) constraints that enforce a principled trade-off between compression and task relevance, while also encouraging cross-modal consistency. Because MI is intractable for high-dimensional visual representations, we instantiate InfoTok with practical, differentiable dependence estimators, including a variational IB formulation and a Hilbert Schmidt Independence Criterion (HSIC) based alternative. Integrated into three representative unified MLLMs without introducing any additional training data, InfoTok consistently improves both image understanding and generation performance. These results support information-regularized visual tokenization as a sound basis for token learning in unified MLLMs.
AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
Institute of Medical Technology, Peking University Health Science Center(北京大学医学部医学技术研究所);Department of Biomedical Engineering, Peking University(北京大学生物医学工程系);National Biomedical Imaging Center, Peking University(北京大学国家生物医学成像中心)
Grounding Everything in Tokens for Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence(人工智能混合专家模型关键实验室);AI Institute, Shanghai Jiao Tong University(上海交通大学人工智能研究院);Central Research Institute, Huawei(华为中央研究院)
GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));Great Bay University(大湾区大学);Dongguan Key Laboratory for Intelligence and Information Technology(东莞市智能信息技术重点实验室)
EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoning
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University(电子与电气工程系,香港理工大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电子与电气工程学院,南洋理工大学);Shien-Ming Wu School of Intelligent Engineering, South China University of Technology(智能工程学院,华南理工大学)
From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities
School of Computer Science, Peking University(北京大学计算机学院);The Chinese University of Hong Kong(香港中文大学);University of California, San Diego(加利福尼亚大学圣迭戈分校);Beijing Academy of Artificial Intelligence(北京人工智能研究院);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所)
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University(西安交通大学人工智能与机器人研究所);Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd(星辰通用人工智能实验室,中国电信人工智能技术(北京)有限公司);University of Science and Technology Beijing(北京科技大学);Beijing University of Posts and Telecommunications(北京邮电大学);Shanghai Jiao Tong University(上海交通大学)
Token Communication in the Era of Large Models: An Information Bottleneck-Based Approach
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(网络与交换技术国家重点实验室,北京邮电大学);Department of Mathematics and Theories, Peng Cheng Laboratory(数学与理论部,鹏城实验室);Department of Electronic Engineering, Tsinghua University(电子工程系,清华大学);Pervasive Communications Center, Purple Mountain Laboratories(感知计算中心,紫金山实验室);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)