In recent years, open-vocabulary (OV) object detection has attracted increasing research attention. Unlike traditional detection, which only recognizes fixed-category objects, OV detection aims to detect objects in an open category set. Previous works often leverage vision-language (VL) training data (e.g., referring grounding data) to recognize OV objects. However, they only use pairs of nouns and individual objects in VL data, while these data usually contain much more information, such as scene graphs, which are also crucial for OV detection. In this paper, we propose a novel Scene-Graph-Based Discovery Network (SGDN) that exploits scene graph cues for OV detection. Firstly, a scene-graph-based decoder (SGDecoder) including sparse scene-graph-guided attention (SSGA) is presented. It captures scene graphs and leverages them to discover OV objects. Secondly, we propose scene-graph-based prediction (SGPred), where we build a scene-graph-based offset regression (SGOR) mechanism to enable mutual enhancement between scene graph extraction and object localization. Thirdly, we design a cross-modal learning mechanism in SGPred. It takes scene graphs as bridges to improve the consistency between cross-modal embeddings for OV object classification. Experiments on COCO and LVIS demonstrate the effectiveness of our approach. Moreover, we show the ability of our model for OV scene graph detection, while previous OV scene graph generation methods cannot tackle this task.
Expanding Scene Graph Boundaries: Fully Open-vocabulary Scene Graph Generation via Visual-Concept Alignment and Retention
The Hong Kong Polytechnic University(香港理工大学);Centre for Artificial Intelligence and Robotics, HKISI-CAS(香港中文大学(深圳)人工智能与机器人中心(HKISI-CAS));State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所 多模态人工智能系统国家重点实验室);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学 人工智能学院)
O2V-Mapping: Online Open-Vocabulary Mapping with Neural Implicit Representation
Academy for Engineering & Technology(工程与技术学院);Fudan University(复旦大学);State Key Laboratory of Intelligent Vehicle Safety Technology(智能车辆安全技术国家重点实验室);School of Future Technology(未来技术学院);Harbin Institute of Technology(哈尔滨工业大学);Department of Computer Science and Engineering(计算机科学与工程系);Shanghai Jiao Tong University(上海交通大学)
Unveiling the Unknown: Open Vocabulary Object Detection with Scene Graphs
Faculty of Electrical Engineering and Computer Science, Ningbo University(宁波大学电气工程与计算机科学学院);Faculty of Computing, Georg-August-Universität Göttingen(哥廷根大学计算机学院);Merchants’ Guild Economics and Cultural Intelligent Computing Laboratory, Ningbo University(宁波大学商帮经济与文化智能计算实验室);School of Software Technology, Zhejiang University(浙江大学软件学院)
Open-Vocabulary vs Supervised Learning Methods for Post-Disaster Visual Scene Understanding
Department of Informatics and Telematics, Harokopio University of Athens(信息与电信学系,雅典哈罗科波斯大学);Department of Networks and Digital Media, Kingston University(网络与数字媒体系,金士顿大学);Department of Electrical and Computer Engineering, University of Western Macedonia(电子与计算机工程系,西马其顿大学);Archimedes, Athena Research Center(阿基米德,雅典研究中心)
Bridging 3D Gaussians and Semantic Occupancy for Comprehensive Open-Vocabulary Scene Understanding from Unposed Images
The Hong Kong Polytechnic University(香港理工大学);Eastern Institute of Technology(东方理工大学);Shanghai Jiao Tong University(上海交通大学);Wuhan University(武汉大学);Great Wall Motor(长城汽车);Tsinghua University(清华大学);Ningbo Institute of Digital Twin, Eastern Institute of Technology(东方理工大学宁波数字孪生研究院)
Nanyang Technological University(南洋理工大学);Max Planck Institute for Informatics(马克斯·普朗克信息学研究所);University of Ottawa(渥太华大学);Carnegie Mellon University(卡内基梅隆大学);MBZUAI(穆罕默德·本·扎耶德人工智能大学)
SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation
Nanyang Technological University(南洋理工大学);MoE Key Laboratory of Interdisciplinary Research of Computation and Economics, Shanghai University of Finance and Economics(上海财经大学经济与计算交叉研究重点实验室);Institute for Infocomm Research (I 2 R), A*STAR, Singapore(新加坡资讯与通讯研究院)
Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding
State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS)(多模态人工智能系统国家重点实验室);Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Information Science and Technology, ShanghaiTech University(上海科技大学信息科学与技术学院);The Chinese University of Hong Kong(香港中文大学);Deepeleph Intelligent Technology(深智科技)
School of Electronic Information, Wuhan University of Science and Technology(武汉科技大学电子信息学院);State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications(北京邮电大学网络与交换技术国家重点实验室);Oriental Space Port Research Institute(东方航天港研究院);School of Artificial Intelligence, Wuhan University(武汉大学人工智能学院)
OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation
University of Cambridge(剑桥大学);The University of Hong Kong(香港大学);Hong Kong University of Science and Technology(香港科技大学);The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Global-Local Collaborative Inference with LLM for Lidar-Based Open-Vocabulary Detection
Institute of Artificial Intelligence, Beihang University(北京航空航天大学人工智能研究院);Meituan(美团);University of Science and Technology Beijing(北京科技大学);Hangzhou Innovation Institute, Beihang University(北京航空航天大学杭州创新研究院)
NoOVD: Novel Category Discovery and Embedding for Open-Vocabulary Object Detection
College of Intelligence and Computing, Tianjin University(天津大学智能与计算学院);Key Research Center for Surface Monitoring and Analysis of Relics, State Administration of Cultural Heritage(文物表面监测与分析国家重点研究中心);Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology(深圳先进技术大学计算机科学与人工智能学院);School of Artificial Intelligence, Nanchang University(南昌大学人工智能学院)
Depth-Only Open-Vocabulary 3D Semantic Segmentation For Privacy-Preserving Robotic Applications
Humanoid Robots Lab, University of Bonn(波恩大学仿人机器人实验室);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习和人工智能研究所);Center for Robotics, Bonn, Germany(波恩机器人中心)
OVGaussian: Generalizable 3D Gaussian Segmentation with Open Vocabularies
The University of Sydney(悉尼大学);Fudan University(复旦大学);Shanghai AI Laboratory(上海人工智能实验室);The University of Hong Kong(香港大学);National University of Singapore(新加坡国立大学);Imperial College London(伦敦帝国学院);The University of Melbourne(墨尔本大学);Texas A&M University(德克萨斯农工大学)
Open-Vocabulary 3D Object Detection with Co-Distillation Discovery and Dual Guidance Robust Training
School of Computer Science and Engineering, University of Electronic Science and Technology of China(电子科技大学计算机科学与工程学院);Singapore University of Technology and Design(新加坡科技设计大学);School of Computer Science and Technology, Hainan University(海南大学计算机科学与技术学院)