Open-vocabulary learning has emerged as a cutting-edge research area, particularly in light of the widespread adoption of vision-based foundational models. Its primary objective is to comprehend novel concepts that are not encompassed within a predefined vocabulary. One key facet of this endeavor is Visual Grounding, which entails locating a specific region within an image based on a corresponding language description. While current foundational models excel at various visual language tasks, there's a noticeable absence of models specifically tailored for open-vocabulary visual grounding. This research endeavor introduces novel and challenging OV tasks, namely Open-Vocabulary Visual Grounding and Open-Vocabulary Phrase Localization. The overarching aim is to establish connections between language descriptions and the localization of novel objects. To facilitate this, we have curated a comprehensive annotated benchmark, encompassing 7,272 OV-VG images and 1,000 OV-PL images. In our pursuit of addressing these challenges, we delved into various baseline methodologies rooted in existing open-vocabulary object detection, VG, and phrase localization frameworks. Surprisingly, we discovered that state-of-the-art methods often falter in diverse scenarios. Consequently, we developed a novel framework that integrates two critical components: Text-Image Query Selection and Language-Guided Feature Attention. These modules are designed to bolster the recognition of novel categories and enhance the alignment between visual and linguistic information. Extensive experiments demonstrate the efficacy of our proposed framework, which consistently attains SOTA performance across the OV-VG task. Additionally, ablation studies provide further evidence of the effectiveness of our innovative models. Codes and datasets will be made publicly available at https://github.com/cv516Buaa/OV-VG.
Visual Programming for Zero-shot Open-Vocabulary 3D Visual Grounding
The Future Network of Intelligence Institute（未来智能网络研究院）；The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；School of Science and Engineering, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）理工学院）；IHPC, A*STAR, Singapore（新加坡科研局智能系统研究所）；The University of Hong Kong（香港大学）
Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Interdisciplinary Institute of Artificial Intelligence, Xidian University（西安电子科技大学跨学科人工智能研究院）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Northwest Institute of Nuclear Technology（西北核技术研究所）；School of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）
Context Disentangling and Prototype Inheriting for Robust Visual Grounding
Nanjing University of Science and Technology（南京理工大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；SenseTime Research（商汤科技研究院）
Pengcheng Laboratory（鹏城实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting
Beijing Institute of Technology（北京理工大学）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Shenzhen MSU-BIT University（深圳北理莫斯科大学）；University of Rochester（罗切斯特大学）
