We propose LangHOPS, the first Multimodal Large Language Model (MLLM) based framework for open-vocabulary object-part instance segmentation. Given an image, LangHOPS can jointly detect and segment hierarchical object and part instances from open-vocabulary candidate categories. Unlike prior approaches that rely on heuristic or learnable visual grouping, our approach grounds object-part hierarchies in language space. It integrates the MLLM into the object-part parsing pipeline to leverage its rich knowledge and reasoning capabilities, and link multi-granularity concepts within the hierarchies. We evaluate LangHOPS across multiple challenging scenarios, including in-domain and cross-dataset object-part instance segmentation, and zero-shot semantic segmentation. LangHOPS achieves state-of-the-art results, surpassing previous methods by 5.5% Average Precision (AP) (in-domain) and 4.8% (cross-dataset) on the PartImageNet dataset and by 2.5% mIOU on unseen object parts in ADE20K (zero-shot). Ablation studies further validate the effectiveness of the language-grounded hierarchy and MLLM driven part query refinement strategy. The code will be released here.
ExpAlign: Expectation-Guided Vision-Language Alignment for Open-Vocabulary Grounding
Department of Automation, Tsinghua University, China（清华大学自动化系）；University of Electronic Science and Technology of China, China（电子科技大学）；University of Copenhagen, Denmark（哥本哈根大学）；Lingsu Lab, China（灵素实验室）
NoOVD: Novel Category Discovery and Embedding for Open-Vocabulary Object Detection
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）；Key Research Center for Surface Monitoring and Analysis of Relics, State Administration of Cultural Heritage（文物表面监测与分析国家重点研究中心）；Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology（深圳先进技术大学计算机科学与人工智能学院）；School of Artificial Intelligence, Nanchang University（南昌大学人工智能学院）
SAM-MI: A Mask-Injected Framework for Enhancing Open-Vocabulary Semantic Segmentation with SAM
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)（多模态人工智能系统国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Technical University of Munich（慕尼黑工业大学）；NVIDIA（英伟达）
Recent advances in 3D semantic segmentation with deep neural networks have shown remarkable success, with rapid performance increase on available datasets. However, current 3D semantic segmentation benchmarks contain only a small number of categories -- less than 30 for ScanNet and SemanticKITTI, for instance, which are not enough to reflect the diversity of real environments (e.g., semantic image understanding covers hundreds to thousands of classes). Thus, we propose to study a larger vocabulary for 3D semantic segmentation with a new extended benchmark on ScanNet data with 200 class categor…
From Semantics, Scene to Instance-awareness: Distilling Foundation Model for Grounded Open-vocabulary Situation Recognition
National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Huazhong University of Science and Technology（华中科技大学）；The Hong Kong Polytechnic University（香港理工大学）
The Australian National University（澳大利亚国立大学）；Commonwealth Scientific and Industrial Research Organization（联邦科学与工业研究组织）；Inception Institute of Artificial Intelligence（Inception人工智能研究所）
Conventional object detection models require large amounts of training data. In comparison, humans can recognize previously unseen objects by merely knowing their semantic description. To mimic similar behaviour, zero-shot object detection aims to recognize and localize 'unseen' object instances by using only their semantic information. The model is first trained to learn the relationships between visual and semantic domains for seen objects, later transferring the acquired knowledge to totally unseen objects. This setting gives rise to the need for correct alignment between visual and semanti…
Clutt3R-Seg: Sparse-view 3D Instance Segmentation for Language-grounded Grasping in Cluttered Scenes
Dept. of Mechanical Engineering, SNU（机械工程系，首尔国立大学）；Interdisciplinary Program in Artificial Intelligence, SNU（人工智能跨学科项目，首尔国立大学）；Robotics Lab, Hyundai Motor Company（Hyundai Motor Company机器人实验室）
Grounded GUI Understanding for Vision-Based Spatial Intelligent Agent: Exemplified by Extended Reality Apps
The Chinese University of Hong Kong（香港中文大学）；Harbin Institute of Technology（哈尔滨工业大学）；Southern University of Science and Technology（南方科技大学）；The Hong Kong University of Science and Technology（香港科学大学）
University of Mississippi（密西西比大学）；National University of Singapore（新加坡国立大学）；Independent Researcher（独立研究员）；ASUS Intelligent Cloud Services (AICS)（华硕智能云服务（AICS））
Microsoft（微软）；University of Washington（华盛顿大学）；Meta AI；UCLA（加州大学洛杉矶分校）
We present GLIPv2, a grounded VL understanding model, that serves both localization tasks (e.g., object detection, instance segmentation) and Vision-Language (VL) understanding tasks (e.g., VQA, image captioning). GLIPv2 elegantly unifies localization pre-training and Vision-Language Pre-training (VLP) with three pre-training tasks: phrase grounding as a VL reformulation of the detection task, region-word contrastive learning as a novel region-word level contrastive learning task, and the masked language modeling. This unification not only simplifies the previous multi-stage VLP procedure but …
State Key Laboratory of Precision Blasting, Jianghan University（江汉大学精密爆破重点实验室）；Harvard AI and Robotics Lab, Harvard University（哈佛大学AI与机器人实验室）；School of EIC, Huazhong University of Science and Technology（华中科技大学电子信息学院）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Department of Computer Science, Hong Kong Baptist University（香港 Baptist 大学计算机科学系）；School of Mathematics and Statistics, Hubei University of Education（湖北省教育学院数学与统计学学院）
CRCV University of Central Florida（中佛罗里达大学计算机视觉研究中心）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
The problem of grounding VQA tasks has seen an increased attention in the research community recently, with most attempts usually focusing on solving this task by using pretrained object detectors. However, pre-trained object detectors require bounding box annotations for detecting relevant objects in the vocabulary, which may not always be feasible for real-life large-scale applications. In this paper, we focus on a more relaxed setting: the grounding of relevant visual entities in a weakly supervised manner by training on the VQA task alone. To address this problem, we propose a visual capsu…
AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $α$-Corrected Binary Cross Entropy and Factorized Latent Supervision
Peter Munk Cardiac Centre, University Health Network (UHN)（彼得·芒克心脏中心，大学健康网络（UHN））；Ted Rogers Centre for Heart Research（泰德·罗杰斯心脏研究中心）；University of Toronto (U of T)（多伦多大学）；Vector Institute（向量研究所）；Toronto General Hospital Research Institute, UHN（多伦多总医院研究所，大学健康网络（UHN））
Jet Propulsion Laboratory（喷气推进实验室）；California Institute of Technology（加州理工学院）；Swiss Federal Institute of Technology（瑞士联邦理工学院）；ETH Zürich（苏黎世联邦理工学院）；FieldAI Inc.（FieldAI公司）；Stanford University（斯坦福大学）；University of California Berkeley（加州大学伯克利分校）
UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interaction
NLPR, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）；Peking University（北京大学）；Tsinghua University（清华大学）；Galbot Inc.（北京银河通用机器人有限公司）；Shanghai Jiao Tong University（上海交通大学）；Beijing Academy of Artificial Intelligence（北京智源人工智能研究院）
A Visual Semantic Adaptive Watermark grounded by Prefix-Tuning for Large Vision-Language Model
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Zhejiang University（浙江大学）
