The availability of large-scale remote sensing video data underscores the importance of high-quality interactive segmentation. However, challenges such as small object sizes, ambiguous features, and limited generalization make it difficult for current methods to achieve this goal. In this work, we propose ROS-SAM, a method designed to achieve high-quality interactive segmentation while preserving generalization across diverse remote sensing data. The ROS-SAM is built upon three key innovations: 1) LoRA-based fine-tuning, which enables efficient domain adaptation while maintaining SAM's generalization ability, 2) Enhancement of deep network layers to improve the discriminability of extracted features, thereby reducing misclassifications, and 3) Integration of global context with local boundary details in the mask decoder to generate high-quality segmentation masks. Additionally, we design the data pipeline to ensure the model learns to better handle objects at varying scales during training while focusing on high-quality predictions during inference. Experiments on remote sensing video datasets show that the redesigned data pipeline boosts the IoU by 6%, while ROS-SAM increases the IoU by 13%. Finally, when evaluated on existing remote sensing object tracking datasets, ROS-SAM demonstrates impressive zero-shot capabilities, generating masks that closely resemble manual annotations. These results confirm ROS-SAM as a powerful tool for fine-grained segmentation in remote sensing applications. Code is available at https://github.com/ShanZard/ROS-SAM.
Training a modern deep neural network on massive labeled samples is the main paradigm in solving the scene classification problem for remote sensing, but learning from only a few data points remains a challenge. Existing methods for few-shot remote sensing scene classification are performed in a sample-level manner, resulting in easy overfitting of learned features to individual samples and inadequate generalization of learned category segmentation surfaces. To solve this problem, learning should be organized at the task level rather than the sample level. Learning on tasks sampled from a task…
This work develops a novel end-to-end deep unsupervised learning method based on convolutional neural network (CNN) with pseudo-classes for remote sensing scene representation. First, we introduce center points as the centers of the pseudo classes and the training samples can be allocated with pseudo labels based on the center points. Therefore, the CNN model, which is used to extract features from the scenes, can be trained supervised with the pseudo labels. Moreover, a pseudo-center loss is developed to decrease the variance between the samples and the corresponding pseudo center point. The …
University of Zurich（苏黎世大学）；University of Twente（特文特大学）；University of Trento（特伦托大学）
The success of supervised classification of remotely sensed images acquired over large geographical areas or at short time intervals strongly depends on the representativity of the samples used to train the classification algorithm and to define the model. When training samples are collected from an image (or a spatial region) different from the one used for mapping, spectral shifts between the two distributions are likely to make the model fail. Such shifts are generally due to differences in acquisition and atmospheric conditions or to changes in the nature of the object observed. In order t…
Scene classification is a key problem in the interpretation of high-resolution remote sensing imagery. Many state-of-the-art methods, e.g. bag-of-visual-words model and its variants, the topic models as well as deep learning-based approaches, share similar procedures: patch sampling, feature description/learning and classification. Patch sampling is the first and a key procedure which has a great influence on the results. In the literature, many different sampling strategies have been used, {e.g. dense sampling, random sampling, keypoint-based sampling and saliency-based sampling, etc. However…
The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot
University of Western São Paulo (UNOESTE)（圣保罗西部大学）；University of Tennessee（田纳西大学）；Federal University of Mato Grosso do Sul (UFMS)（南马托格罗索联邦大学）；São Paulo State University (UNESP)（圣保罗州立大学）；University of Waterloo（滑铁卢大学）
本文探索了 Meta AI 的 Segment Anything Model (SAM) 在遥感图像分割中的应用，通过结合文本提示与单样本训练的自动化技术提升了多尺度数据上的分割精度，并开源代码以推动 SAM 在遥感领域的进一步适配。
Deep learning techniques have achieved great success in remote sensing image change detection. Most of them are supervised techniques, which usually require large amounts of training data and are limited to a particular application. Self-supervised methods as an unsupervised approach are popularly used to solve this problem and are widely used in unsupervised binary change detection tasks. However, the existing self-supervised methods in change detection are based on pre-tasks or at patch-level, which may be sub-optimal for pixel-wise change detection tasks. Therefore, in this work, a pixel-wi…
With the development of deep learning, supervised learning methods perform well in remote sensing images (RSIs) scene classification. However, supervised learning requires a huge number of annotated data for training. When labeled samples are not sufficient, the most common solution is to fine-tune the pre-training models using a large natural image dataset (e.g. ImageNet). However, this learning paradigm is not a panacea, especially when the target remote sensing images (e.g. multispectral and hyperspectral data) have different imaging mechanisms from RGB natural images. To solve this problem…
TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentation
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Wuhan University（武汉大学）；Dalian University of Technology（大连理工大学）；University of Chinese Academy of Sciences（中国科学院大学）
In this paper, we deal with the problem of object detection on remote sensing images. Previous methods have developed numerous deep CNN-based methods for object detection on remote sensing images and the report remarkable achievements in detection performance and efficiency. However, current CNN-based methods mostly require a large number of annotated samples to train deep neural networks and tend to have limited generalization abilities for unseen object categories. In this paper, we introduce a few-shot learning-based method for object detection on remote sensing images where only a few anno…
College of Electronic Science and Technology, National University of Defense Technology（国防科技大学电子科学学院）
Change detection of heterogeneous remote sensing images is an important and challenging topic in remote sensing for emergency situation resulting from nature disaster. Due to the different imaging mechanisms of heterogeneous sensors, it is difficult to directly compare the images. To address this challenge, we explore an unsupervised change detection method based on adaptive local structure consistency (ALSC) between heterogeneous images in this letter, which constructs an adaptive graph representing the local structure for each patch in one image domain and then projects this graph to the oth…
SAM2-ELNet: Label Enhancement and Automatic Annotation for Remote Sensing Segmentation
College of Instrumentation and Electrical Engineering, Jilin University（吉林大学仪器电气工程学院）；Key Laboratory of Geophysical Exploration Equipment, Ministry of Education（教育部地球物理探测装备重点实验室）；School of Mathematics, Jilin University（吉林大学数学学院）
Prompt-Calibrated SAM 3 for Open-Vocabulary Remote Sensing Semantic Segmentation
School of Information Science and Technology, Yunnan Normal University（云南师范大学信息科学与技术学院）；School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；College of Computer Science, Beijing University of Technology（北京工业大学计算机学院）
Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences（中国科学院遥感与数字地球研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Lack of annotated samples greatly restrains the direct application of deep learning in remote sensing image scene classification. Although researches have been done to tackle this issue by data augmentation with various image transformation operations, they are still limited in quantity and diversity. Recently, the advent of the unsupervised learning based generative adversarial networks (GANs) bring us a new way to generate augmented samples. However, such GAN-generated samples are currently only served for training GANs model itself and for improving the performance of the discriminator in G…
RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation
College of Information Science and Technology, Nanjing Forestry University（南京林业大学信息科学技术学院）；Shenzhen Research Institute, Nanjing University of Aeronautics and Astronautics（南京航空航天大学深圳研究院）
Shenzhen University（深圳大学）；Shenzhen Institute of AI and Robotics for Society（深圳人工智能与社会机器人研究院）；Pengcheng Laboratory（鹏城实验室）
Image clustering is a very useful technique that is widely applied to various areas, including remote sensing. Recently, visual representations by self-supervised learning have greatly improved the performance of image clustering. To further improve the well-trained clustering models, this paper proposes a novel method by first ranking samples within each cluster based on the confidence in their belonging to the current cluster and then using the ranking to formulate a weighted cross-entropy loss to train the model. For ranking the samples, we developed a method for computing the likelihood of…
Automatically finding good and general remote sensing representations allows to perform transfer learning on a wide range of applications - improving the accuracy and reducing the required number of training samples. This paper investigates development of generic remote sensing representations, and explores which characteristics are important for a dataset to be a good source for representation learning. For this analysis, five diverse remote sensing datasets are selected and used for both, disjoint upstream representation learning and downstream model training and evaluation. A common evaluat…
Self-supervised methods have shown tremendous success in the field of computer vision, including applications in remote sensing and medical imaging. Most popular contrastive-loss based methods like SimCLR, MoCo, MoCo-v2 use multiple views of the same image by applying contrived augmentations on the image to create positive pairs and contrast them with negative examples. Although these techniques work well, most of these techniques have been tuned on ImageNet (and similar computer vision datasets). While there have been some attempts to capture a richer set of deformations in the positive sampl…
