The University of Western Australia(西澳大利亚大学);Murdoch University(莫道克大学);Sun Yat-sen University(中山大学);The Hong Kong University of Science and Technology(香港科技大学)
Lian Xu, Mohammed Bennamoun, Farid Boussaid, Hamid Laga, Yulan Guo, Dan Xu
Referring expression comprehension (REC) aims to localize the object in an image described by natural language. In Weakly supervised REC (WREC), existing approaches primarily operate on anchor-level visual representations. Even when enriched with auxiliary cues, relational interactions remain implicitly encoded within individual anchor features. The resulting visual representation remains flat and unary-only, limiting its ability to align with the structured nature of language. In this work, we propose a Structured Visual Compositional Representation (SVCR) learning framework for WREC. Rather than implicitly encoding relations within unary anchors, the proposed SVCR explicitly models both unary object embeddings and pairwise relational embeddings, forming a structured visual representation space. We further introduce a compositional alignment mechanism that matches unary and pairwise visual representations with their corresponding textual embeddings in a unified manner, enabling compositional visual-textual matching under weak supervision. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that the proposed SVCR achieves state-of-the-art performance. These results demonstrate the effectiveness of explicit structured visual representations and visual-textual alignment for WREC.
The University of Hong Kong(香港大学);University of Wollongong(伍伦贡大学);Tencent AI Lab(腾讯人工智能实验室);Australian Centre for Robotic Vision, University of Adelaide(阿德莱德大学澳大利亚机器人视觉中心)
Northwestern Polytechnical University(西北工业大学);National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology(空天地海一体化大数据应用技术国家工程实验室)
Referring Expression Comprehension Using Language Adaptive Inference
College of Computer Science & Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);Shanghai Institute for Advanced Study, Zhejiang University(浙江大学上海高等研究院);Shanghai AI Laboratory(上海人工智能实验室)
Hierarchical Alignment-enhanced Adaptive Grounding Network for Generalized Referring Expression Comprehension
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Institute of Big Data, Fudan University(复旦大学大数据学院);Shanghai University of Finance and Economics(上海财经大学);Nanyang Technological University(南洋理工大学)
ScanFormer: Referring Expression Comprehension by Iteratively Scanning
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);School of Software Technology, Zhejiang University(浙江大学软件学院);Zhejiang-Singapore Innovation and Al Joint Research Lab(浙新创新与人工智能联合研究实验室)
Meta Compositional Referring Expression Segmentation
Singapore University of Technology and Design(新加坡科技设计大学);ByteDance(字节跳动);Institute for Infocomm Research(资讯通信研究院);Centre for Frontier AI Research(前沿人工智能研究中心)
Continual Referring Expression Comprehension via Dual Modular Memorization
University of Electronic Science and Technology of China(电子科技大学);Peng Cheng Laboratory(鹏城实验室);University of Wollongong(伍伦贡大学);Hefei University of Technology(合肥工业大学)