Training-free video understanding leverages the strong image comprehension capabilities of pre-trained vision language models (VLMs) by treating a video as a sequence of static frames, thus obviating the need for costly video-specific training. However, this paradigm often suffers from severe visual redundancy and high computational overhead, especially when processing long videos. Crucially, existing keyframe selection strategies, especially those based on CLIP similarity, are prone to biases and may inadvertently overlook critical frames, resulting in suboptimal video comprehension. To address these significant challenges, we propose \textbf{KTV}, a novel two-stage framework for efficient and effective training-free video understanding. In the first stage, KTV performs question-agnostic keyframe selection by clustering frame-level visual features, yielding a compact, diverse, and representative subset of frames that mitigates temporal redundancy. In the second stage, KTV applies key visual token selection, pruning redundant or less informative tokens from each selected keyframe based on token importance and redundancy, which significantly reduces the number of tokens fed into the LLM. Extensive experiments on the Multiple-Choice VideoQA task demonstrate that KTV outperforms state-of-the-art training-free baselines while using significantly fewer visual tokens, \emph{e.g.}, only 504 visual tokens for a 60-min video with 10800 frames, achieving $44.8\%$ accuracy on the MLVU-Test benchmark. In particular, KTV also exceeds several training-based approaches on certain benchmarks.
Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior
Harbin Institute of Technology (Shenzhen)(哈尔滨工业大学(深圳));Shenzhen Loop Area Institute(深圳河套学院);University of Chinese Academy of Sciences(中国科学院大学);The Hong Kong University of Science and Technology(香港科技大学)
Gaussian Mixture Modeling for Event-Aware Visual Allocation in Long Video Understanding
Beijing Key Laboratory of Super Intelligent Security of Multi-Modal Information, CASIA(中国科学院自动化所多模态信息超智能安全北京市重点实验室);State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA(中国科学院自动化所多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Hello Group(未知(保留英文));School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院)
VisCache: Visual KV Cache Pruning for Efficient Vision Large Language Model Inference
Shenzhen International Center for Industrial and Applied Mathematics(深圳国际工业与应用数学中心);Shenzhen Research Institute of Big Data(深圳大数据研究院);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Loop Area Institute(深圳河套学院)
Less is More: Token-Efficient Video-QA via Adaptive Frame-Pruning and Semantic Graph Integration
The Hong Kong University of Science and Technology (Guangzhou), China(香港科技大学(广州)中国);The Hong Kong University of Science and Technology, Hong Kong SAR, China(香港科技大学,香港特别行政区,中国)
SportsGPT: An LLM-driven Framework for Interpretable Sports Motion Assessment and Training Guidance
School of Artificial Intelligence, UCAS, Beijing, China(人工智能学院);MAIS, NLPR, Institute of Automation, CAS, Beijing, China(自动化研究所);College of Education, Beijing Sport University, Beijing, China(北京体育大学);KuTi Sports Technology, Shaanxi, China(KuTi体育科技);Haoxiong Technology, Shanghai, China(浩鸿科技)
BoxTuning: Directly Injecting the Object Box for Multimodal Model Fine-Tuning
School of Computer Science and Technology, Tianjin University(天津大学计算机科学与技术学院);Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology(深圳理工大学计算机科学与人工智能学院)