Multimodal Large Language Models (MLLMs) have shown strong performance on video question answering, but their application to long-form videos is constrained by limited context length and computational cost, making keyframe sampling essential. Existing approaches typically rely on semantic relevance or reinforcement learning, which either fail to capture evidential clues or suffer from inefficient combinatorial optimization. In this work, we propose an evidence-driven keyframe sampling framework grounded in information bottleneck theory. We formulate keyframe selection as maximizing the conditional mutual information between selected frames and the query, providing a principled objective that reflects each frame's contribution to answering the question. To make this objective tractable, we exploit its structure to derive a decomposed optimization that reduces subset selection to independent frame-level scoring. We further introduce a query-conditioned evidence scoring network trained with a contrastive objective to estimate evidential importance efficiently. Experiments on long-form video understanding benchmarks show that our method consistently outperforms prior sampling strategies under strict token budgets, while significantly improving training efficiency.
Reasoning with Memory: A Temporal Granularity-Adaptive Framework for Training-Free Long Video Understanding
National University of Singapore(新加坡国立大学);Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)(高性能计算研究所,科学、技术与研究机构(A*STAR));Nanyang Technological University(南洋理工大学);University of Science and Technology of China(中国科学技术大学);Jilin University(吉林大学)
FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding
Georgia Institute of Technology(佐治亚理工学院);Center for Signal and Information Processing (CSIP)(信号与信息处理中心 (CSIP));School of Electrical and Computer Engineering(电气与计算机工程学院)
BoxTuning: Directly Injecting the Object Box for Multimodal Model Fine-Tuning
School of Computer Science and Technology, Tianjin University(天津大学计算机科学与技术学院);Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology(深圳理工大学计算机科学与人工智能学院)
Less is More: Token-Efficient Video-QA via Adaptive Frame-Pruning and Semantic Graph Integration
The Hong Kong University of Science and Technology (Guangzhou), China(香港科技大学(广州)中国);The Hong Kong University of Science and Technology, Hong Kong SAR, China(香港科技大学,香港特别行政区,中国)
GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding
Huazhong University of Science and Technology(华中科技大学);University of Notre Dame(圣母大学);Imperial College London(伦敦帝国学院);Peking University(北京大学);Microsoft Research(微软研究院);Lehigh University(理海大学)