Referring Image Segmentation (RIS) aims at segmenting the target object from an image referred by one given natural language expression. The diverse and flexible expressions as well as complex visual contents in the images raise the RIS model with higher demands for investigating fine-grained matching behaviors between words in expressions and objects presented in images. However, such matching behaviors are hard to be learned and captured when the visual cues of referents (i.e. referred objects) are insufficient, as the referents with weak visual cues tend to be easily confused by cluttered background at boundary or even overwhelmed by salient objects in the image. And the insufficient visual cues issue can not be handled by the cross-modal fusion mechanisms as done in previous work. In this paper, we tackle this problem from a novel perspective of enhancing the visual information for the referents by devising a Two-stage Visual cues enhancement Network (TV-Net), where a novel Retrieval and Enrichment Scheme (RES) and an Adaptive Multi-resolution feature Fusion (AMF) module are proposed. Through the two-stage enhancement, our proposed TV-Net enjoys better performances in learning fine-grained matching behaviors between the natural language expression and image, especially when the visual information of the referent is inadequate, thus produces better segmentation results. Extensive experiments are conducted to validate the effectiveness of the proposed method on the RIS task, with our proposed TV-Net surpassing the state-of-the-art approaches on four benchmark datasets.
Progressive Prompt-Guided Cross-Modal Reasoning for Referring Image Segmentation
School of Computer Science and Artificial Intelligence, Wuhan University of Technology(武汉理工大学计算机科学与人工智能学院);School of Artificial Intelligence and Automation, Huazhong University of Science and Technology(华中科技大学人工智能与自动化学院);University of Electronic Science and Technology of China(电子科技大学);Wuhan Vocational College of Software and Engineering(武汉软件工程职业学院)
CausalCLIPSeg: Unlocking CLIP's Potential in Referring Medical Image Segmentation with Causal Intervention
Wuhan University of Technology(武汉理工大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);MedAI Technology (Wuxi) Co. Ltd.(MedAI科技(无锡)有限公司);Technical University of Munich(慕尼黑工业大学)
EviRCOD: Evidence-Guided Probabilistic Decoding for Referring Camouflaged Object Detection
West China Hospital, Sichuan University(四川大学华西医院);Independent Researcher(独立研究员);Department of Data Science and Artificial Intelligence, Auckland University of Technology(奥克兰理工大学数据科学与人工智能系)
Confidence-guided Adaptive Gate and Dual Differential Enhancement for Video Salient Object Detection
Sun Yat-sen University(中山大学);Guangdong Province Key Laboratory of Information Security Technology(广东省信息安全技术重点实验室);Key Laboratory of Machine Intelligence and Advanced Computing, Ministry of Education(教育部机器智能与先进计算重点实验室)
Cross-Modal Progressive Comprehension for Referring Segmentation
Beihang University(北京航空航天大学);Chinese Academy of Sciences(中国科学院);University of Chinese Academy of Sciences(中国科学院大学);Beijing Jiaotong University(北京交通大学);Sun Yat-sen University(中山大学);Pazhou Lab(琶洲实验室)
Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation
School of Information Science and Technology, Southwest Jiaotong University(西南交通大学信息科学与技术学院);Ant Group(蚂蚁集团);Luoyang Institute of Electro-optical Equipment, AVIC(中国航空工业集团公司洛阳电光设备研究所);Faculty of Geosciences and Engineering, Southwest Jiaotong University(西南交通大学地球科学与环境工程学院)
Vision-Aware Text Features in Referring Image Segmentation: From Object Understanding to Context Understanding
The Hong Kong University of Science and Technology(香港科技大学);Stony Brook University(石溪大学);University of Science, VNU-HCM(胡志明市国家大学科学大学);Vietnam National University, Ho Chi Minh City(胡志明市国家大学);Trinity College Dublin(都柏林大学学院)
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University(上海交通大学信息科学与工程学院);College of Computer Science, Nankai University(南开大学计算机学院);Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际研究生院);Department of Electronic Engineering, Tsinghua University(清华大学电子工程系)
RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China(教育部多媒体可信感知与高效计算重点实验室);Institute of Artificial Intelligence, Xiamen University(厦门大学人工智能研究院);School of Informatics, Xiamen University(厦门大学信息学院)
RefAtomNet++: Advancing Referring Atomic Video Action Recognition using Multi-Trajectory Semantic Retrieval
Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology(人机化研究所,卡尔斯鲁厄技术大学);RISE Research Institutes of Sweden(瑞典RISE研究机构);KTH Royal Institute of Technology(皇家理工学院);School of Artificial Intelligence and Robotics(人工智能与机器人学院);National Engineering Research Center of Robot Visual Perception and Control Technology(机器人视觉感知与控制技术国家工程研究中心);Chinese University of Hong Kong(香港中文大学);Shanghai AI Lab(上海人工智能实验室)
Efficient Language-to-Vision Feature Injection for Referring Single-Object Tracking
Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences(中国科学院空间应用工程与技术中心);University of Chinese Academy of Sciences(中国科学院大学)
Entity-enhanced Adaptive Reconstruction Network for Weakly Supervised Referring Expression Grounding
University of Chinese Academy of Sciences(中国科学院大学);Peng Cheng Laboratory(鹏城实验室);University of Science and Technology of China(中国科学技术大学);Nanjing University of Science and Technology(南京理工大学);Huawei Cloud & AI(华为云与人工智能部门)
The University of Western Australia(西澳大利亚大学);Murdoch University(莫道克大学);Sun Yat-sen University(中山大学);The Hong Kong University of Science and Technology(香港科技大学)
OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentation
MEGVII Technology(旷视科技);Shanghai Jiao Tong University(上海交通大学);Beijing Academy of Artificial Intelligence(北京智源人工智能研究院);Beijing Institute of Technology(北京理工大学);University of Macau(澳门大学)
Referring Multi-Object Tracking with Diverse Dynamic Expressions
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University(计算机科学学院、多媒体软件国家工程研究中心、人工智能研究院、湖北省多媒体与网络通信工程重点实验室、武汉大学);Multimedia Lab (MMLab), Department of Information Engineering, the Chinese University of Hong Kong(多媒体实验室(MMLab)、信息工程系、香港中文大学);State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, University of Macau(智能城市物联网国家重点实验室、计算机与信息科学系、澳门大学);College of Information Technology, United Arab Emirates University(信息科技学院、阿联酋大学)