Multimodal Large Language Models (MLLMs) face significant computational overhead when processing long videos due to the massive number of visual tokens required. To improve efficiency, existing methods primarily reduce redundancy by pruning or merging tokens based on importance or similarity. However, these approaches largely overlook a critical dimension of video content, i.e., changes and turning points, and they lack a collaborative model for spatio-temporal relationships. To address this, we propose a new perspective: similarity is for identifying redundancy, while difference is for capturing key events. Based on this, we designed a training-free framework named ST-SimDiff. We first construct a spatio-temporal graph from the visual tokens to uniformly model their complex associations. Subsequently, we employ a parallel dual-selection strategy: 1) similarity-based selection uses community detection to retain representative tokens, compressing static information; 2) temporal difference-based selection precisely locates content-changing points to preserve tokens that capture key dynamic shifts. This allows it to preserve both static and dynamic content with a minimal number of tokens. Extensive experiments show our method significantly outperforms state-of-the-art approaches while substantially reducing computational costs. Our code is available in https://github.com/bingjunluo/ST-SimDiff.
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling
Shanghai AI Laboratory(上海人工智能实验室);Nanjing University(南京大学);Shanghai Innovation Institute(上海创新研究院);Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(深圳先进技术研究院,中国科学院)
A Survey on Video Temporal Grounding with Multimodal Large Language Model
School of Computer Science and Technology, Harbin Institute of Technology(哈尔滨工业大学计算机科学与技术学院);School of Computer Science and Technology, Shandong Jianzhu University(山东建筑大学计算机科学与技术学院);School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院);Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系)
EventVL: Understand Event Streams via Multimodal Large Language Model
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));KU Leuven(根特大学);École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Carleton University(卡尔顿大学)
Mind over Space: Can Multimodal Large Language Models Mentally Navigate?
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);Dept. of Comp. Sci. and Tech., Institute for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University(清华大学-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学计算机科学与技术系);School of Automation Science and Electrical Engineering , Beihang University(北京航空航天大学自动化科学与电气工程学院);college of AI, Tsinghua University(清华大学人工智能学院);Department of Computer Science and Technology , Tsinghua University(清华大学计算机科学与技术系)
Evaluating point-light biological motion in multimodal large language models
Psychiatry and Biobehavioral Sciences, UCLA(乌尔拉克大学精神病学与生物行为科学系);Brain and Creativity Institute, USC(美国大学脑与创造力研究所);Google DeepMind, Zurich(谷歌深度Mind瑞士分公司)
MotionAtlas: Detailed Region Captioning for Motion-Centric Videos
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);Shanghai Jiao Tong University(上海交通大学);Nanyang Technological University(南洋理工大学);Peking University(北京大学);Wuhan University(武汉大学);Beijing Jiaotong University(北京交通大学)
SpaR3D-MoE: Adaptive 3D Spatial Reasoning from Sparse Views Meets Geometry-Inductive Mixture-of-Experts
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);YUKUN Intelligent World(宇琨智能世界)
Geospatial-Temporal Sensemaking of Remote Sensing Activity Detections with Multimodal Large Language Model
SenSIP Center, School of ECEE, Arizona State University(SenSIP中心,电子与计算机工程学院,亚利桑那州立大学);Prime Solutions Group Inc(Prime Solutions Group公司);Intelligence Advanced Research Projects Activity(智能高级研究计划局)
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Nanyang Technological University(南洋理工大学);University of California, Berkeley(加州大学伯克利分校);University of California, San Diego(加州大学圣地亚哥分校);The University of Tokyo(东京大学);The Chinese University of Hong Kong(香港中文大学);University of Michigan(密歇根大学);Johns Hopkins University(约翰霍普金斯大学);University of California, Davis(加州大学戴维斯分校);University of California, Los Angeles(加州大学洛杉矶分校);Carnegie Mellon University(卡内基梅隆大学);Columbia University(哥伦比亚大学);University of Toronto(多伦多大学)