In multimodal large language models (MLLMs), inference cost is largely dominated by the visual token prefix rather than the language backbone, making token reduction a key factor for improving efficiency. Existing approaches typically assign independent importance scores to visual tokens and retain a fixed number of top-ranked tokens, implicitly assuming token independence and a uniform compression ratio across inputs. In this work, we reformulate visual token pruning as a sequential decision-making process. Specifically, we introduce a pointer-style selection mechanism that iteratively chooses informative tokens, conditioning each decision on previously selected ones, and dynamically determines when to stop via a learned termination action. This enables joint optimization of both the selected subset and its size. To enable end-to-end training under standard language modeling objectives, we design a differentiable relaxation based on a variance-preserving noise interpolation scheme, allowing gradients to propagate through the discrete selection process. Extensive experiments on LLaVA-v1.5-7B and Qwen2.5-VL-7B demonstrate that our approach consistently outperforms fixed-ratio baselines across different compression levels. Under aggressive pruning that removes 88.9% of visual tokens, our method preserves 94.6% of the original accuracy while achieving a 1.88x speed-up in prefill latency.
Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction
School of Software Technology, Zhejiang University（浙江大学软件学院）；Zhejiang Key Laboratory of Digital-Intelligence Service Technology（浙江省数字化服务技术重点实验室）；Hong Kong Polytechnic University（香港理工大学）；The University of Southern Queensland（南昆士兰大学）
Spotlight on Token Perception for Multimodal Reinforcement Learning
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；The Chinese University of Hong Kong（香港中文大学）；Nanjing University（南京大学）；Wuhan University（武汉大学）
Grounding Everything in Tokens for Multimodal Large Language Models
MoE Key Lab of Artificial Intelligence（人工智能混合专家模型关键实验室）；AI Institute, Shanghai Jiao Tong University（上海交通大学人工智能研究院）；Central Research Institute, Huawei（华为中央研究院）
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd（星辰通用人工智能实验室，中国电信人工智能技术（北京）有限公司）；University of Science and Technology Beijing（北京科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Jiao Tong University（上海交通大学）
Latent Noise Mask for Reducing Visual Redundancy in Multimodal Large Language Models
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University（人工智能学院、光学和电子学（iOPEN）、西北工业大学）；Institute of Artificial Intelligence, China Telecom (TeleAI)（人工智能研究院、中国电信（TeleAI））；Fudan University（复旦大学）；The University of Hong Kong（香港大学）
MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs
Arizona State University（亚利桑那州立大学）；University of Washington（华盛顿大学）；University of Texas at Dallas（德克萨斯大学达拉斯分校）；Duke University（杜克大学）；Zoom Communications（Zoom通讯）
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China（东北大学计算机科学与工程学院，沈阳 110819，中国）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China（东北大学计算机与通信工程学院，秦皇岛 066004，中国）
SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism
Shenzhen Institute for Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳研究院）；Southwestern University of Finance and Economics（西南财经大学）；Engineering Research Center of Intelligent Finance, Ministry of Education（教育部智能金融工程研究中心）；Tongji University（同济大学）
AdaTok: Adaptive Token Compression with Object-Aware Representations for Efficient Multimodal LLMs
Institute of Medical Technology, Peking University Health Science Center（北京大学医学部医学技术研究所）；Department of Biomedical Engineering, Peking University（北京大学生物医学工程系）；National Biomedical Imaging Center, Peking University（北京大学国家生物医学成像中心）
Multimodal Stepwise Clinically-Guided Attention Learning for Pathological Complete Response Prediction in Breast Cancer
Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma（人工智能与计算机系统单位，工程系，罗马生物医学大学）；Department of Electrical Engineering and Information Technology, University of Naples Federico II（电气工程与信息科技系，那不勒斯费德里科二世大学）；Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University（诊断与介入系，辐射物理，生物医学工程，乌梅大学）
We present an effective method for fusing visual-and-language representations for several question answering tasks including visual question answering and visual entailment. In contrast to prior works that concatenate unimodal representations or use only cross-attention, we compose multimodal representations via channel fusion. By fusing on the channels, the model is able to more effectively align the tokens compared to standard methods. These multimodal representations, which we call compound tokens are generated with cross-attention transformer layers. First, vision tokens are used as querie…
Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；Zhongguancun Academy（中关村学院）；University of Chinese Academy of Sciences（中国科学院大学）；Nanyang Technological University（南洋理工大学）；Department of Mechanical Engineering, Imperial College London（伦敦帝国理工学院机械工程系）
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Sun Yat-sen University（中山大学）；National Supercomputing Center in Shenzhen（国家超级计算深圳中心）；Tsinghua University（清华大学）
