Video Large Language Models (Video-LLMs) achieve strong performance in video understanding, but their excessive visual tokens bring substantial computational overhead. Existing training-free compression methods improve inference efficiency by reducing visual tokens, yet they often rely on local adjacent-frame similarity for temporal redundancy estimation or allocate token budgets mainly according to segment length. Such designs are sensitive to frame-level noise and fail to capture the non-uniform information distribution of real-world videos. To address these challenges, we propose InfoMerge, a training-free visual token compression method that improves token utilization through robust redundancy estimation and content-aware budget allocation. Specifically, we propose the Temporal Fingerprint Difference: a segment-level second-order temporal redundancy estimation strategy, which models the temporal similarity structure of tokens at the same spatial positions within each segment. We further introduce Content-Aware Budget Allocation (CABA), which dynamically allocates segment-level token budgets based on segment uniqueness and spectral-entropy-based representational richness. By reducing repeated preservation of redundant static regions and allocating more tokens to informative segments, InfoMerge makes better use of the limited token budget while maintaining strong performance. Extensive experiments show that InfoMerge achieves strong efficiency--accuracy trade-offs across multiple benchmarks and backbones, with more pronounced advantages under aggressive compression. On LLaVA-OneVision-7B, InfoMerge retains 98.8\% of the original average performance while reducing 85\% of visual tokens and achieving a 4.24-fold speedup in the prefill stage.
OmniSIFT: Modality-Asymmetric Token Compression for Efficient Omni-modal Large Language Models
New Laboratory of Pattern Recognition (NLPR), Institute of Automation, Chinese Academy of Sciences (CASIA)(模式识别新实验室(NLPR)、自动化研究所、中国科学院(CASIA));Nanjing University(南京大学);The Hong Kong University of Science(香港科学大学);Sichuan University(四川大学);Peking University(北京大学)
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models
NLP Lab, School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院自然语言处理实验室);NiuTrans Research, Shenyang, China(NiuTrans研究院)
Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Shenzhen Loop Area Institute(深圳河套学院);University of Chinese Academy of Sciences(中国科学院大学);The Hong Kong University of Science and Technology(香港科技大学)
TCRA-LLM: Token Compression Retrieval Augmented Large Language Model for Inference Cost Reduction
Meetyou AI Lab(美柚人工智能实验室);Xiamen Key Laboratory of Women’s Internet Health Management(厦门市女性互联网健康管理重点实验室);Osaka University(大阪大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局)
Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors
School of Computer Science and Engineering, Northeastern University, Shenyang, China(东北大学计算机科学与工程学院,中国沈阳);NiuTrans Research, Shenyang, China(沈阳NiuTrans研究院,中国);Kunming University of Science and Technology, Kunming, China(昆明理工大学,中国昆明)
TREX: Tokenizer Regression for Optimal Data Mixture
KAIST CT(韩国科学技术院CT);Seoul National University of Science and Technology(首尔科学技术大学);Upstage AI;KAIST InnoCORE PRISM-AI Center(韩国科学技术院InnoCORE PRISM-AI中心)
MultiTok: Variable-Length Tokenization for Efficient LLMs Adapted from LZW Compression
University of Texas at Austin(德克萨斯大学奥斯汀分校);Nokia Bell Labs(诺基亚贝尔实验室);Boğaziçi University(博雅大学);Georgia Institute of Technology(佐治亚理工学院);Massachusetts Institute of Technology (MIT)(麻省理工学院(MIT))
Entropy Gate: Entropy Quenching for Near-Lossless Token Compression in LLM Pipelines
VIA Cybersecurity Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana(维亚网络安全实验室,卡布雷姆大学科学与技术学院,库马西,加纳);Quantum and Assistive Technologies Lab, Kwame Nkrumah University of Science and Technology, Kumasi, Ghana(量子与辅助技术实验室,卡布雷姆大学科学与技术学院,库马西,加纳)