Video Large Language Models (VLLMs) demonstrate strong video understanding but suffer from inefficiency due to redundant visual tokens. Existing pruning primary targets intra-frame spatial redundancy or prunes inside the LLM with shallow-layer overhead, yielding suboptimal spatiotemporal reduction and underutilizing long-context compressibility. All of them often discard subtle yet informative context from merged or pruned tokens. In this paper, we propose a new perspective that elaborates token \textbf{A}nchors within intra-frame and inter-frame to comprehensively aggregate the informative contexts via local-global \textbf{O}ptimal \textbf{T}ransport (\textbf{AOT}). Specifically, we first establish local- and global-aware token anchors within each frame under the attention guidance, which then optimal transport aggregates the informative contexts from pruned tokens, constructing intra-frame token anchors. Then, building on the temporal frame clips, the first frame within each clip will be considered as the keyframe anchors to ensemble similar information from consecutive frames through optimal transport, while keeping distinct tokens to represent temporal dynamics, leading to efficient token reduction in a training-free manner. Extensive evaluations show that our proposed AOT obtains competitive performances across various short- and long-video benchmarks on leading video LLMs, obtaining substantial computational efficiency while preserving temporal and visual fidelity. Project webpage: https://tyroneli.github.io/AOT.
Video-KTR: Reinforcing Video Reasoning via Key Token Attribution
ByteDance(字节跳动);School of Electrical and Electronic Engineering, Nanyang Technological University(南洋理工大学电子与电气工程学院);National University of Singapore(国立新加坡大学);College of Computing and Data Science, Nanyang Technological University(南洋理工大学计算与数据科学学院)
Token Merging via Spatiotemporal Information Mining for Surgical Video Understanding
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology(电子与计算机工程系,香港科学与技术大学);School of Electronic Information and Communications, Huazhong University of Science and Technology(电子信息与通信学院,华中科技大学)
Accelerating Streaming Video Large Language Models via Hierarchical Token Compression
EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Sichuan University(四川大学);Huazhong University of Science and Technology(华中科技大学);Sun Yat-sen University(中山大学);Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Seeing the Forest and the Trees: Query-Aware Tokenizer for Long-Video Multimodal Language Models
Queen Mary University of London(伦敦女王学院);University of Sheffield(谢菲尔德大学);Imperial College London(伦敦帝国学院);Pengcheng Laboratory(鹏城实验室);Meta Inc(Meta公司);Meituan Inc(美团公司);Nanjing University of Science(南京理工大学);University of Birmingham(伯明翰大学);Utrecht University(乌得勒支大学)
Beyond Entropy: Self-Diagnostic Multi-Role Token Optimization for Video Reasoning
City University of Hong Kong(香港城市大学);Institute of Information Engineering, Chinese Academy of Sciences (CAS)(中国科学院信息工程研究所);Beihang University(北京航空航天大学);AMAP, Alibaba Group(阿里巴巴集团高德地图);Computer Science, Beijing Institute of Technology(北京理工大学计算机学院)
From Pixels to Privacy: Temporally Consistent Video Anonymization via Token Pruning for Privacy Preserving Action Recognition
Aalborg University(奥尔堡大学);Aarhus University(奥胡斯大学);Poineer Centre for AI(先锋人工智能中心);University of Southern Denmark(南丹麦大学);Milestone System(Milestone系统公司)
MovieChat: From Dense Token to Sparse Memory for Long Video Understanding
University of Washington(华盛顿大学);Microsoft Research Asia(微软亚洲研究院);Hong Kong University of Science and Technology (GZ)(香港科技大学(广州));Donghua University(东华大学);Zhejiang University(浙江大学)
V-CAST: Video Curvature-Aware Spatio-Temporal Pruning for Efficient Video Large Language Models
Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区);Shenzhen Loop Area Institute(深圳河套学院);Sichuan University(四川大学);EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室)
School of Computer Science, Peking University(北京大学计算机学院);Beijing Academy of Artificial Intelligence(北京人工智能研究院);University of California, San Diego(加利福尼亚大学圣迭戈分校);Renmin University of China(中国人民大学)
CoVisco: Codec-Native Vision Encoder with Native Token Compression for Unified Image-Video Understanding
ERNIE Team, Baidu Inc.(百度公司ERNIE团队);The Hong Kong University of Science and Technology(香港科技大学);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所)