Referring remote sensing image segmentation is crucial for achieving fine-grained visual understanding through free-format textual input, enabling enhanced scene and object extraction in remote sensing applications. Current research primarily utilizes pre-trained language models to encode textual descriptions and align them with visual modalities, thereby facilitating the expression of relevant visual features. However, these approaches often struggle to establish robust alignments between fine-grained semantic concepts, leading to inconsistent representations across textual and visual information. To address these limitations, we introduce a referring remote sensing image segmentation foundational model, RSRefSeg. RSRefSeg leverages CLIP for visual and textual encoding, employing both global and local textual semantics as filters to generate referring-related visual activation features in the latent space. These activated features then serve as input prompts for SAM, which refines the segmentation masks through its robust visual generalization capabilities. Experimental results on the RRSIS-D dataset demonstrate that RSRefSeg outperforms existing methods, underscoring the effectiveness of foundational models in enhancing multimodal task comprehension. The code is available at \url{https://github.com/KyanChen/RSRefSeg}.
Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation
School of Information Science and Technology, Southwest Jiaotong University（西南交通大学信息科学与技术学院）；Ant Group（蚂蚁集团）；Luoyang Institute of Electro-optical Equipment, AVIC（中国航空工业集团公司洛阳电光设备研究所）；Faculty of Geosciences and Engineering, Southwest Jiaotong University（西南交通大学地球科学与环境工程学院）
Intra- and Cross-Scale Feature Interaction Network for Referring Remote Sensing Image Segmentation
School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University（人工智能、光学与电子学院（iOPEN），西北工业大学）；School of Computer Science, Northwestern Polytechnical University（计算机科学学院，西北工业大学）；department of Software, Northwest Institute of Nuclear Technology（软件系，西北核技术研究院）
Image-Conditioned Instance Prompt Network for Referring Remote Sensing Image Segmentation
School of Computer Science, Northwestern Polytechnical University, Xi'an, China（西北工业大学计算机科学学院，西安，中国）；College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China（复旦大学智能机器人与先进制造学院，上海，中国）；Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, China（西北工业大学深圳研究院，深圳，中国）
CADFormer: Fine-Grained Cross-modal Alignment and Decoding Transformer for Referring Remote Sensing Image Segmentation
School of Computer Science and Technology, Wuhan University of Science and Technology（武汉科技大学计算机科学与技术学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（教育部多媒体可信感知与高效计算重点实验室）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；Xiamen University（厦门大学）；School of Informatics, Xiamen University（厦门大学信息学院）
Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation
School of Electronics and Information Engineering, Harbin Institute of Technology（电子信息工程学院，哈尔滨工业大学）；Faculty of Computing, Harbin Institute of Technology（计算机学院，哈尔滨工业大学）；Peng Cheng Laboratory, Shenzhen（鹏城实验室，深圳）
GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation
School of Internet, Anhui University（安徽大学互联网学院）；National Engineering Research Center for Agro-Ecological Big Data Analysis & Application, Anhui University（安徽大学农业生态大数据分析与应用国家工程研究中心）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（教育部多媒体可信感知与高效计算重点实验室）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；School of Informatics, Xiamen University（厦门大学信息学院）
Faculty of Mathematics and Computer Science（数学与计算机科学学院）
In recent years, the geospatial industry has been developing at a steady pace. This growth implies the addition of satellite constellations that produce a copious supply of satellite imagery and other Remote Sensing data on a daily basis. Sometimes, this information, even if in some cases we are referring to publicly available data, it sits unaccounted for due to the sheer size of it. Processing such large amounts of data with the help of human labour or by using traditional automation methods is not always a viable solution from the standpoint of both time and other resources. Within the pres…
Nanjing University of Information Technology（南京信息工程大学）
As the rapid development of depth learning, object detection in aviatic remote sensing images has become increasingly popular in recent years. Most of the current Anchor Free detectors based on key point detection sampling directly regression and classification features, with the design of object loss function based on the horizontal bounding box. It is more challenging for complex and diverse aviatic remote sensing object. In this paper, we propose an Anchor Free aviatic remote sensing object detector (BWP-Det) to detect rotating and multi-scale object. Specifically, we design a interactive d…
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
Household robots operate in the same space for years. Such robots incrementally build dynamic maps that can be used for tasks requiring remote object localization. However, benchmarks in robot learning often test generalization through inference on tasks in unobserved environments. In an observed environment, locating an object is reduced to choosing from among all object proposals in the environment, which may number in the 100,000s. Armed with this intuition, using only a generic vision-language scoring model with minor modifications for 3d encoding and operating in an embodied environment, …
The University of Adelaide（阿德莱德大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
One of the long-term challenges of robotics is to enable robots to interact with humans in the visual world via natural language, as humans are visual animals that communicate through language. Overcoming this challenge requires the ability to perform a wide variety of complex tasks in response to multifarious instructions from humans. In the hope that it might drive progress towards more flexible and powerful human interactions with robots, we propose a dataset of varied and complex robot tasks, described in natural language, in terms of objects visible in a large set of real images. Given an…
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction. When facing such a situation, a human tends to imagine what the destination may look like and to explore the environment based on prior knowledge of the environmental layout, such as the fact that a bathroom is more likely to be found near a bedroom than a kitchen. We have designed an autonomous agent called Layout-aware Dreamer (LAD), including two novel mo…
Prompting DirectSAM for Semantic Contour Extraction in Remote Sensing Images
College of Computer Science and Software Engineering, Hohai University（河海大学计算机与软件学院）；Hong Kong University of Science and Technology（香港科技大学）；School of Computer and Electronic Information, Nanjing Normal University（南京师范大学计算机与电子信息学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
The University of Hong Kong（香港大学）；Sun Yat-sen University（中山大学）；Deepwise AI Lab（深度思考人工智能实验室）
Humans learn from life events to form intuitions towards the understanding of visual environments and languages. Envision that you are instructed by a high-level instruction, "Go to the bathroom in the master bedroom and replace the blue towel on the left wall", what would you possibly do to carry out the task? Intuitively, we comprehend the semantics of the instruction to form an overview of where a bathroom is and what a blue towel is in mind; then, we navigate to the target location by consistently matching the bathroom appearance in mind with the current scene. In this paper, we present an…
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Mohamed bin Zayed University of AI（穆罕默德·本·扎耶德人工智能大学）；Birla Institute of Technology & Science, Hyderabad（海得拉巴贝拉理工学院）；Australian National University（澳大利亚国立大学）；Linköping University（林雪平大学）
Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos
School of Computer Science and Technology, Xidian University（西安电子科技大学计算机科学与技术学院）；Interdisciplinary Institute of Artificial Intelligence, Xidian University（西安电子科技大学跨学科人工智能研究院）；School of Artificial Intelligence, Xidian University（西安电子科技大学人工智能学院）；Northwest Institute of Nuclear Technology（西北核技术研究所）；School of Physics and Information Engineering, Fuzhou University（福州大学物理与信息工程学院）
EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting
School of Interdisciplinary Science, Beijing Institute of Technology（交叉科学学院，北京理工大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）；National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology（空间智能信息处理国家重点实验室，北京理工大学）；School of Optics and Photonics, Beijing Institute of Technology（光学与 photonics 学院，北京理工大学）；State Key Laboratory of Explosion Science and Safety Protection, Beijing（爆炸科学与安全防护国家重点实验室，北京）
