SynthRef: Generation of Synthetic Referring Expressions for Object Segmentation
Universitat Politècnica de Catalunya(加泰罗尼亚理工大学);National Technical University of Athens(雅典国立技术大学);Universitat Oberta de Catalunya(加泰罗尼亚开放大学);Barcelona Supercomputing Center(巴塞罗那超级计算中心);University of Stuttgart(斯图加特大学)
Ioannis Kazakos, Carles Ventura, Miriam Bellver, Carina Silberer, Xavier Giro-i-Nieto
Recent advances in deep learning have brought significant progress in visual grounding tasks such as language-guided video object segmentation. However, collecting large datasets for these tasks is expensive in terms of annotation time, which represents a bottleneck. To this end, we propose a novel method, namely SynthRef, for generating synthetic referring expressions for target objects in an image (or video frame), and we also present and disseminate the first large-scale dataset with synthetic referring expressions for video object segmentation. Our experiments demonstrate that by training with our synthetic referring expressions one can improve the ability of a model to generalize across different datasets, without any additional annotation cost. Moreover, our formulation allows its application to any object detection or segmentation dataset.
Proxy-Embedding as an Adversarial Teacher: An Embedding-Guided Bidirectional Attack for Referring Expression Segmentation Models
National Pilot School of Software, Yunnan University(云南大学软件试点学校);School of Information Science and Engineering, Yunnan University(云南大学信息科学与工程学院);Institute of Data Science, National University of Singapore(新加坡国立大学数据科学研究所)
RefVOS: A Closer Look at Referring Expressions for Video Object Segmentation
Barcelona Supercomputing Center(巴塞罗那超级计算中心);Universitat Oberta de Catalunya(加泰罗尼亚开放大学);Institute for NLP, University of Stuttgart(斯图加特大学自然语言处理研究所);Universitat Pompeu Fabra(庞培法布拉大学);National Technical University of Athens(雅典国立技术大学);Universitat Politècnica de Catalunya(加泰罗尼亚理工大学)
UniPixel: Unified Object Referring and Segmentation for Pixel-Level Visual Reasoning
The Hong Kong Polytechnic University(香港理工大学);ARC Lab, Tencent PCG(腾讯PCG ARC实验室);Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);vivo Mobile Communication Co.(vivo移动通信公司);MindWingman Technology (Shenzhen) Co., Ltd.(深圳MindWingman技术有限公司)
EventRR: Event Referential Reasoning for Referring Video Object Segmentation
Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Hong Kong University of Science and Technology (Guangzhou)(香港科学与技术大学(广州));The Hong Kong University of Science and Technology(香港科学与技术大学);Northwestern Polytechnical University(西北工业大学)
Meta Compositional Referring Expression Segmentation
Singapore University of Technology and Design(新加坡科技设计大学);ByteDance(字节跳动);Institute for Infocomm Research(资讯通信研究院);Centre for Frontier AI Research(前沿人工智能研究中心)
RefSAM: Efficiently Adapting Segmenting Anything Model for Referring Video Object Segmentation
College of Computer Science and Technology, National University of Defense Technology(国防科技大学计算机学院);School of Computer Science, The University of Sydney(悉尼大学计算机学院)
Learning to Label: A Reinforced Self-Evolving Framework for Semi-supervised Referring Expression Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);School of Information Engineering, Huzhou Normal University(湖州师范学院信息工程学院);School of Artificial Intelligence, Nanjing University of Posts and Telecommunications(南京邮电大学人工智能学院);National Key Laboratory of Tibetan Language Intelligence(藏语智能国家重点实验室);Zhejiang University(浙江大学);Beijing Normal University(北京师范大学)
Event-Aware Instructed Assistant for Referring Video Segmentation
Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与技术学院大数据研究所);Institute of Trustworthy Embodied AI, Fudan University(复旦大学可信具身智能研究所);Shanghai University of Finance and Economics(上海财经大学)
A Unified Mutual Supervision Framework for Referring Expression Segmentation and Generation
The Chinese University of Hong Kong(香港中文大学);International Digital Economy Academy (IDEA)(国际数字经济研究院(IDEA));The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学)
指称表达式理解和切分是两个高度相关的任务,它们都是为了根据自然语言表达式识别指称对象。在本文中,我们提出了一种新的多任务协作网络(MCN),以实现联合学习的REC和RES的第一次。在MCN中,RES可以帮助REC实现更好的语言-视觉对齐,而REC可以帮助RES更好地定位所指对象。此外,我们解决了这种多任务设置中的一个关键挑战,即,预测冲突,两个创新的设计,即,一致性能量最大化(CEM)和自适应软非定位抑制(ASNLS)。具体来说,CEM使REC和RES能够通过最大化两个任务之间的一致性能量来关注相似的视觉区域。ASNLS基于REC的预测来抑制RES中不相关区域的响应。为了验证我们的模型,我们在REC和RES的三个基准数据集上进行了广泛的实验,即,RefCOCO、RefCOCO+和RefCOCOg。实验结果报告了MCN在所有现有方法上的显着性能增益,即,最高可达+7.13%for REC和+11.50%for RES over SOTA,这充分证实了我们的模型对于REC和RES联合学习的有效性。
DRAgent: Discriminative Reasoning Agent for Referring Expression Segmentation
School of Computer Science and Technology, Hangzhou Dianzi University(杭州电子科技大学计算机科学与技术学院);Khoury College of Computer Sciences, Northeastern University(东北大学Khoury计算机科学学院)
IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation
Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Ningbo Global Innovation Center, Zhejiang University(浙江大学宁波全球创新中心);Zhejiang Key Laboratory of Digital-Intelligence Service Technology(浙江省数字智能服务技术重点实验室)
SSP-SAM: SAM with Semantic-Spatial Prompt for Referring Expression Segmentation
School of Computer Science and Engineering, Nanjing University of Science and Technology(南京理工大学计算机科学与工程学院);NExT++ Lab, School of Computing, National University of Singapore(新加坡国立大学计算机学院NExT++实验室);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所)
MVGGT: Multimodal Visual Geometry Grounded Transformer for Multiview 3D Referring Expression Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(厦门大学多媒体可信感知与高效计算教育部重点实验室);Shanghai Innovation Institute(上海创新研究院);Fudan University(复旦大学);ByteDance(字节跳动);Tianjin University of Science and Technology(天津科技大学)
Unveiling Parts Beyond Objects:Towards Finer-Granularity Referring Expression Segmentation
Institute of Automation, Chinese Academy of Sciences (CASIA)(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences (UCAS)(中国科学院大学人工智能学院);Beijing Academy of Artificial Intelligence (BAAI)(北京人工智能研究院);University of Science and Technology Beijing (USTB)(北京科技大学)
CoHD: A Counting-Aware Hierarchical Decoding Framework for Generalized Referring Expression Segmentation
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);School of EIC, Huazhong University of Science and Technology(华中科技大学电子信息与工程学院);Tencent TEG(腾讯技术工程事业群)
Cross-Modal Progressive Comprehension for Referring Segmentation
Beihang University(北京航空航天大学);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Beijing Jiaotong University(北京交通大学);Sun Yat-sen University(中山大学);Pazhou Lab(琶洲实验室)
Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China(教育部多媒体可信感知与高效计算重点实验室);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);Xiamen University(厦门大学);School of Informatics, Xiamen University(厦门大学信息学院)