Segmenting long-form videos into semantically coherent scenes is a fundamental task in large-scale video understanding. Existing encoder-based methods are limited by visual-centric biases, classify each shot in isolation without leveraging sequential dependencies, and lack both narrative understanding and explainability. In this paper, we present Scene-VLM, the first fine-tuned vision-language model (VLM) framework for video scene segmentation. Scene-VLM jointly processes visual and textual cues including frames, transcriptions, and optional metadata to enable multimodal reasoning across consecutive shots. The model generates predictions sequentially with causal dependencies among shots and introduces a context-focus window mechanism to ensure sufficient temporal context for each shot-level decision. In addition, we propose a scheme to extract confidence scores from the token-level logits of the VLM, enabling controllable precision-recall trade-offs that were previously limited to encoder-based methods. Furthermore, we demonstrate that our model can be aligned to generate coherent natural-language rationales for its boundary decisions through minimal targeted supervision. Our approach achieves state-of-the-art performance on standard scene segmentation benchmarks. On MovieNet, for example, Scene-VLM yields significant improvements of +6 AP and +13.7 F1 over the previous leading method.
Advancing Complex Wide-Area Scene Understanding with Hierarchical Coresets Selection
Institute of Software Chinese Academy of Sciences(中国科学院软件研究所);University of the Chinese Academy of Sciences(中国科学院大学);Beijing University of Technology(北京理工大学)
Does Your 3D Encoder Really Work? When Pretrain-SFT from 2D VLMs Meets 3D VLMs
Shenzhen campus of Sun Yat-sen University(中山大学深圳校区);Huawei Noah’s Ark Lab(华为诺亚实验室);MBZUAI;Peng Cheng Laboratory(鹏城实验室);Guangdong Key Laboratory of Big Data Analysis and Processing(广东省大数据分析与处理重点实验室)
本文分析以 3D 场景为中心的 VLM 性能受限原因,发现其弱用 3D 编码器、依赖语言捷径,并提出 3D Relevance Discrimination QA 数据集以促进真实 3D 理解。
Anomalous Frame Detection Using VLM-Based Description Comparison for Extracting Expert-Specific Actions and Contextual Decision-Making Scenes with Intra-Video Self-Similarity
VOST-SGG: VLM-Aided One-Stage Spatio-Temporal Scene Graph Generation
College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算与数据科学学院);Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore(科学、技术与研究局高性能计算研究所);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(科学、技术与研究局前沿人工智能研究中心)
HMR3D: Hierarchical Multimodal Representation for 3D Scene Understanding with Large Vision-Language Model
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore(高性能计算研究所,科技研究局,新加坡);Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore(前沿人工智能研究中心,科技研究局,新加坡);College of Computing and Data Science, Nanyang Technological University, Singapore(计算与数据科学学院,南洋理工大学,新加坡)
MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMs
Arizona State University(亚利桑那州立大学);University of Washington(华盛顿大学);University of Texas at Dallas(德克萨斯大学达拉斯分校);Duke University(杜克大学);Zoom Communications(Zoom通讯)
From Pixels to Graphs: Open-Vocabulary Scene Graph Generation with Vision-Language Models
School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);Shanghai AI Laboratory(上海人工智能实验室);Shanghai Engineering Research Center of Intelligent Vision and Imaging(上海智能视觉与成像工程技术研究中心)
RE-VLM: Event-Augmented Vision-Language Model for Scene Understanding
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(北京邮电大学人工智能学院);State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)