Visual grounding, i.e., localizing objects in images according to natural language queries, is an important topic in visual language understanding. The most effective approaches for this task are based on deep learning, which generally require expensive manually labeled image-query or patch-query pairs. To eliminate the heavy dependence on human annotations, we present a novel method, named Pseudo-Q, to automatically generate pseudo language queries for supervised training. Our method leverages an off-the-shelf object detector to identify visual objects from unlabeled images, and then language queries for these objects are obtained in an unsupervised fashion with a pseudo-query generation module. Then, we design a task-related query prompt module to specifically tailor generated pseudo language queries for visual grounding tasks. Further, in order to fully capture the contextual relationships between images and language queries, we develop a visual-language model equipped with multi-level cross-modality attention mechanism. Extensive experimental results demonstrate that our method has two notable benefits: (1) it can reduce human annotation costs significantly, e.g., 31% on RefCOCO without degrading original model's performance under the fully supervised setting, and (2) without bells and whistles, it achieves superior or comparable performance compared to state-of-the-art weakly-supervised visual grounding methods on all the five datasets we have experimented. Code is available at https://github.com/LeapLabTHU/Pseudo-Q.
Weakly-Supervised 3D Visual Grounding based on Visual Language Alignment
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；Meituan Inc.（美团公司）；College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；Guangdong Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）
ViewSRD: 3D Visual Grounding via Structured Multi-View Decomposition
South China University of Technology（华南理工大学）；Guangdong Engineering Center for Large Model and GenAI Technology（广东省大模型与生成式人工智能技术工程中心）；State Key Laboratory of Subtropical Building and Urban Science（亚热带建筑科学国家重点实验室）；Ministry of Education Key Laboratory of Big Data and Intelligent Robot（教育部大数据与智能机器人重点实验室）；Guangdong Provincial Key Lab of Computational Intelligence and Cyberspace Information（广东省计算智能与网络信息重点实验室）；Singapore Management University（新加坡国立大学）
CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Grounding
Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；Peng Cheng Laboratory (PCL)（鹏城实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)（中国科学院大学人工智能学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）
ResVG: Enhancing Relation and Semantic Understanding in Multiple Instances for Visual Grounding
Wangxuan Institute of Computer Technology, Peking University（北京大学王选计算机研究所）；National Institute of Health Data Science, Peking University（北京大学健康数据科学国家研究院）；State Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）
GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）；National University of Defense Technology（国防科技大学）；Shanghai Jiao Tong University（上海交通大学）；Chongqing University（重庆大学）
Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding
East China University of Science and Technology（华东理工大学）；Tsinghua University（清华大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）
UniRef-UAV: A Multimodal Benchmark for Universal Referring in UAV Imagery
School of Automation, Northwestern Polytechnical University（自动化学院，西北工业大学）；College of Missile Engineering, Rocket Force University of Engineering（导弹工程学院，火箭兵工程大学）；School of Artificial Intelligence, Chongqing University of Posts and Telecommunications（人工智能学院，重庆邮电大学）
Uncertainty-aware Medical Diagnostic Phrase Identification and Grounding
College of Computer Science, Sichuan University（四川大学计算机学院）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）；Department of Computing, The Hong Kong Polytechnic University（香港理工大学计算机系）；Department of Radiology, West China Hospital, Sichuan University（四川大学华西医院放射科）；College of Intelligence and Computing, Tianjin University（天津大学智能与计算学院）；Department of Mathematics, Sichuan University（四川大学数学系）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-Sen University（中山大学深圳校区网络科学与技术学院）；Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore and the Singapore Eye Research Institute, Singapore National Eye Centre（新加坡国立大学 Yong Loo Lin 医学院眼科系及新加坡眼科研所、新加坡国家眼科中心）
GeoSearcher: Anchor-Guided Progressive Reasoning for Remote Sensing Visual Grounding with Process Supervision
Key Laboratory of Target Cognition and Application Technology (TCAT), Chinese Academy of Sciences（中国科学院目标认知与应用技术重点实验室）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
Mixture-of-Thought-Tokens: Unifying Perception and Reasoning for Free-form Multimodal Grounding
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University（西安交通大学人工智能与机器人研究所）；Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd（星辰通用人工智能实验室，中国电信人工智能技术（北京）有限公司）；University of Science and Technology Beijing（北京科技大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai Jiao Tong University（上海交通大学）
GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；LLM-Core Xiaomi（小米LLM-Core）；The University of Hong Kong（香港大学）；Renmin University of China（中国人民大学）
