Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation
School of Information Science and Technology, Southwest Jiaotong University（西南交通大学信息科学与技术学院）；Ant Group（蚂蚁集团）；Luoyang Institute of Electro-optical Equipment, AVIC（中国航空工业集团公司洛阳电光设备研究所）；Faculty of Geosciences and Engineering, Southwest Jiaotong University（西南交通大学地球科学与环境工程学院）
Given a language expression, referring remote sensing image segmentation (RRSIS) aims to identify ground objects and assign pixel-wise labels within the imagery. The one of key challenges for this task is to capture discriminative multi-modal features via text-image alignment. However, the existing RRSIS methods use one vanilla and coarse alignment, where the language expression is directly extracted to be fused with the visual features. In this paper, we argue that a ``fine-grained image-text alignment'' can improve the extraction of multi-modal information. To this point, we propose a new referring remote sensing image segmentation method to fully exploit the visual and linguistic representations. Specifically, the original referring expression is regarded as context text, which is further decoupled into the ground object and spatial position texts. The proposed fine-grained image-text alignment module (FIAM) would simultaneously leverage the features of the input image and the corresponding texts, obtaining better discriminative multi-modal representation. Meanwhile, to handle the various scales of ground objects in remote sensing, we introduce a Text-aware Multi-scale Enhancement Module (TMEM) to adaptively perform cross-scale fusion and intersections. We evaluate the effectiveness of the proposed method on two public referring remote sensing datasets including RefSegRS and RRSIS-D, and our method obtains superior performance over several state-of-the-art methods. The code will be publicly available at https://github.com/Shaosifan/FIANet.
Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（教育部多媒体可信感知与高效计算重点实验室）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；Xiamen University（厦门大学）；School of Informatics, Xiamen University（厦门大学信息学院）
Intra- and Cross-Scale Feature Interaction Network for Referring Remote Sensing Image Segmentation
School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University（人工智能、光学与电子学院（iOPEN），西北工业大学）；School of Computer Science, Northwestern Polytechnical University（计算机科学学院，西北工业大学）；department of Software, Northwest Institute of Nuclear Technology（软件系，西北核技术研究院）
Image-Conditioned Instance Prompt Network for Referring Remote Sensing Image Segmentation
School of Computer Science, Northwestern Polytechnical University, Xi'an, China（西北工业大学计算机科学学院，西安，中国）；College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China（复旦大学智能机器人与先进制造学院，上海，中国）；Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, China（西北工业大学深圳研究院，深圳，中国）
CADFormer: Fine-Grained Cross-modal Alignment and Decoding Transformer for Referring Remote Sensing Image Segmentation
School of Computer Science and Technology, Wuhan University of Science and Technology（武汉科技大学计算机科学与技术学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）
DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation
School of Information Science and Electronic Engineering, Shanghai Jiao Tong University（上海交通大学信息科学与工程学院）；College of Computer Science, Nankai University（南开大学计算机学院）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）
Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation
School of Electronics and Information Engineering, Harbin Institute of Technology（电子信息工程学院，哈尔滨工业大学）；Faculty of Computing, Harbin Institute of Technology（计算机学院，哈尔滨工业大学）；Peng Cheng Laboratory, Shenzhen（鹏城实验室，深圳）
GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation
School of Internet, Anhui University（安徽大学互联网学院）；National Engineering Research Center for Agro-Ecological Big Data Analysis & Application, Anhui University（安徽大学农业生态大数据分析与应用国家工程研究中心）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（武汉大学测绘遥感信息工程国家重点实验室）
SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；School of Information and Communications Engineering, Xi’an Jiaotong University（西安交通大学信息与通信工程学院）；Ministry of Education Key Lab of Intelligent Network Security, Xi’an, China（教育部智能网络安全重点实验室）；Pazhou Laboratory (Huangpu), Radarweg 29, Guangzhou, China（琶洲实验室（黄埔），雷达路29号，广州，中国）
RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China（教育部多媒体可信感知与高效计算重点实验室）；Institute of Artificial Intelligence, Xiamen University（厦门大学人工智能研究院）；School of Informatics, Xiamen University（厦门大学信息学院）
Faculty of Mathematics and Computer Science（数学与计算机科学学院）
In recent years, the geospatial industry has been developing at a steady pace. This growth implies the addition of satellite constellations that produce a copious supply of satellite imagery and other Remote Sensing data on a daily basis. Sometimes, this information, even if in some cases we are referring to publicly available data, it sits unaccounted for due to the sheer size of it. Processing such large amounts of data with the help of human labour or by using traditional automation methods is not always a viable solution from the standpoint of both time and other resources. Within the pres…
Exploring DINO: Emergent Properties and Limitations for Synthetic Aperture Radar Imagery
Universidad Nacional de Colombia（哥伦比亚国立大学）；European Space Agency（欧洲空间局）；Universitat de València（瓦伦西亚大学）；University of Cambridge（剑桥大学）；University of Oxford（牛津大学）；Universidad de Antioquia（安蒂奥基亚大学）
National University of Sciences and Technology (NUST)（国立科技大学）；Technical University of Munich (TUM)（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心）
Semantic segmentation of raw 3D point clouds is an essential component in 3D scene analysis, but it poses several challenges, primarily due to the non-Euclidean nature of 3D point clouds. Although, several deep learning based approaches have been proposed to address this task, but almost all of them emphasized on using the latent (global) feature representations from traditional convolutional neural networks (CNN), resulting in severe loss of spatial information, thus failing to model the geometry of the underlying 3D objects, that plays an important role in remote sensing 3D scenes. In this l…
Nanjing University of Information Technology（南京信息工程大学）
As the rapid development of depth learning, object detection in aviatic remote sensing images has become increasingly popular in recent years. Most of the current Anchor Free detectors based on key point detection sampling directly regression and classification features, with the design of object loss function based on the horizontal bounding box. It is more challenging for complex and diverse aviatic remote sensing object. In this paper, we propose an Anchor Free aviatic remote sensing object detector (BWP-Det) to detect rotating and multi-scale object. Specifically, we design a interactive d…
Generalized Few-Shot Semantic Segmentation in Remote Sensing: Challenge and Benchmark
RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所先进智能项目中心）；the University of Tokyo（东京大学）；Ontario Tech University（安大略理工大学）；University of British Columbia（不列颠哥伦比亚大学）
March in Chat: Interactive Prompting for Remote Embodied Referring Expression
Australian Institute for Machine Learning, The University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）
An Integrated Neighborhood and Scale Information Network for Open-Pit Mine Change Detection in High-Resolution Remote Sensing Images
China Aero Geophysical Survey & Remote Sensing Center for Nature Resources, China Geological Survey（中国地质调查局自然资源航空物探遥感中心）；National Engineering Laboratory for Satellite Remote Sensing Applications, Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院遥感卫星应用国家工程实验室）
Household robots operate in the same space for years. Such robots incrementally build dynamic maps that can be used for tasks requiring remote object localization. However, benchmarks in robot learning often test generalization through inference on tasks in unobserved environments. In an observed environment, locating an object is reduced to choosing from among all object proposals in the environment, which may number in the 100,000s. Armed with this intuition, using only a generic vision-language scoring model with minor modifications for 3d encoding and operating in an embodied environment, …
Labeling semantic segmentation datasets is a costly and laborious process if compared with tasks like image classification and object detection. This is especially true for remote sensing applications that not only work with extremely high spatial resolution data but also commonly require the knowledge of experts of the area to perform the manual labeling. Data augmentation techniques help to improve deep learning models under the circumstance of few and imbalanced labeled samples. In this work, we propose a novel data augmentation method focused on exploring the spatial context of remote sens…
The University of Adelaide（阿德莱德大学）；Georgia Institute of Technology（佐治亚理工学院）；University of California, Santa Barbara（加州大学圣巴巴拉分校）
One of the long-term challenges of robotics is to enable robots to interact with humans in the visual world via natural language, as humans are visual animals that communicate through language. Overcoming this challenge requires the ability to perform a wide variety of complex tasks in response to multifarious instructions from humans. In the hope that it might drive progress towards more flexible and powerful human interactions with robots, we propose a dataset of varied and complex robot tasks, described in natural language, in terms of objects visible in a large set of real images. Given an…
Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences（中国科学院西安光机精密机械研究所）；The University of Chinese Academy of Sciences（中国科学院大学）
Inspired by recent development of artificial satellite, remote sensing images have attracted extensive attention. Recently, noticeable progress has been made in scene classification and target detection.However, it is still not clear how to describe the remote sensing image content with accurate and concise sentences. In this paper, we investigate to describe the remote sensing images with accurate and flexible sentences. First, some annotated instructions are presented to better describe the remote sensing images considering the special characteristics of remote sensing images. Second, in ord…
Think and Answer ME: Benchmarking and Exploring Multi-Entity Reasoning Grounding in Remote Sensing
Beihang University, Beijing, China（北京航空航天大学）；University of Liverpool, Liverpool, UK（利物浦大学）；Institute of Acoustics, Chinese Academy of Sciences, Beijing, China（中国科学院声学研究所）
In this work, we study the problem of Embodied Referring Expression Grounding, where an agent needs to navigate in a previously unseen environment and localize a remote object described by a concise high-level natural language instruction. When facing such a situation, a human tends to imagine what the destination may look like and to explore the environment based on prior knowledge of the environmental layout, such as the fact that a bathroom is more likely to be found near a bedroom than a kitchen. We have designed an autonomous agent called Layout-aware Dreamer (LAD), including two novel mo…
Universidad Tecnológica Nacional（国立技术大学）；CONICET（国家科学技术研究委员会）
During recent years transformers architectures have been growing in popularity. Modulated Detection Transformer (MDETR) is an end-to-end multi-modal understanding model that performs tasks such as phase grounding, referring expression comprehension, referring expression segmentation, and visual question answering. One remarkable aspect of the model is the capacity to infer over classes that it was not previously trained for. In this work we explore the use of MDETR in a new task, action detection, without any previous training. We obtain quantitative results using the Atomic Visual Actions dat…
SceneMixer: Exploring Convolutional Mixing Networks for Remote Sensing Scene Classification
College of Engineering（工程学院）；University of Dubai（迪拜大学）；Department of Geography（地理系）；Simon Fraser University（西蒙弗雷泽大学）；Department of Computer Science（计算机科学系）；Tezpur University（泰朱普大学）
