Self-supervised contrastive learning (SSCL) has achieved significant milestones in remote sensing image (RSI) understanding. Its essence lies in designing an unsupervised instance discrimination pretext task to extract image features from a large number of unlabeled images that are beneficial for downstream tasks. However, existing instance discrimination based SSCL suffer from two limitations when applied to the RSI semantic segmentation task: 1) Positive sample confounding issue; 2) Feature adaptation bias. It introduces a feature adaptation bias when applied to semantic segmentation tasks that require pixel-level or object-level features. In this study, We observed that the discrimination information can be mapped to specific regions in RSI through the gradient of unsupervised contrastive loss, these specific regions tend to contain singular ground objects. Based on this, we propose contrastive learning with Gradient guided Sampling Strategy (GraSS) for RSI semantic segmentation. GraSS consists of two stages: Instance Discrimination warm-up (ID warm-up) and Gradient guided Sampling contrastive training (GS training). The ID warm-up aims to provide initial discrimination information to the contrastive loss gradients. The GS training stage aims to utilize the discrimination information contained in the contrastive loss gradients and adaptively select regions in RSI patches that contain more singular ground objects, in order to construct new positive and negative samples. Experimental results on three open datasets demonstrate that GraSS effectively enhances the performance of SSCL in high-resolution RSI semantic segmentation. Compared to seven baseline methods from five different types of SSCL, GraSS achieves an average improvement of 1.57\% and a maximum improvement of 3.58\% in terms of mean intersection over the union. The source code is available at https://github.com/GeoX-Lab/GraSS
Self-supervised methods have shown tremendous success in the field of computer vision, including applications in remote sensing and medical imaging. Most popular contrastive-loss based methods like SimCLR, MoCo, MoCo-v2 use multiple views of the same image by applying contrived augmentations on the image to create positive pairs and contrast them with negative examples. Although these techniques work well, most of these techniques have been tuned on ImageNet (and similar computer vision datasets). While there have been some attempts to capture a richer set of deformations in the positive sampl…
ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data
Lane Dept. of Computer Science and Electrical Engineering（兰恩计算机科学与电气工程系）；Dept. of Geology & Geography（地质与地理系）；West Virginia University（西弗吉尼亚大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Green University of Bangladesh（孟加拉国绿色大学）；Stamford University Bangladesh（班加罗尔斯坦福大学）
Multi-level Cross-modal Feature Alignment via Contrastive Learning towards Zero-shot Classification of Remote Sensing Image Scenes
Henan Key Laboratory of Big Data Analysis and Processing（河南省大数据分析与处理重点实验室）；Henan Engineering Laboratory of Spatial Information Processing（河南省空间信息处理工程实验室）；Henan Industrial Technology Academy of Spatio-Temporal Big Data, Henan University（河南大学河南时空大数据产业技术研究院）
Central South University（中南大学）；Wuhan University（武汉大学）；Southwest Jiaotong University（西南交通大学）
Supervised learning for semantic segmentation requires a large number of labeled samples, which is difficult to obtain in the field of remote sensing. Self-supervised learning (SSL), can be used to solve such problems by pre-training a general model with a large number of unlabeled images and then fine-tuning it on a downstream task with very few labeled samples. Contrastive learning is a typical method of SSL that can learn general invariant features. However, most existing contrastive learning methods are designed for classification tasks to obtain an image-level representation, which may be…
Balanced Hierarchical Contrastive Learning with Decoupled Queries for Fine-grained Object Detection in Remote Sensing Images
School of Computer Science and Engineering, Nanjing University of Science and Technology, China（南京理工大学计算机科学与工程学院）；College of Computer Science, Zhejiang University, China（浙江大学计算机学院）
Deep learning techniques have achieved great success in remote sensing image change detection. Most of them are supervised techniques, which usually require large amounts of training data and are limited to a particular application. Self-supervised methods as an unsupervised approach are popularly used to solve this problem and are widely used in unsupervised binary change detection tasks. However, the existing self-supervised methods in change detection are based on pre-tasks or at patch-level, which may be sub-optimal for pixel-wise change detection tasks. Therefore, in this work, a pixel-wi…
In the training of deep learning models, how the model parameters are initialized greatly affects the model performance, sample efficiency, and convergence speed. Representation learning for model initialization has recently been actively studied in the remote sensing field. In particular, the appearance characteristics of the imagery obtained using the a synthetic aperture radar (SAR) sensor are quite different from those of general electro-optical (EO) images, and thus representation learning is even more important in remote sensing domain. Motivated from contrastive multiview coding, we pro…
Contrastive learning methods have significantly narrowed the gap between supervised and unsupervised learning on computer vision tasks. In this paper, we explore their application to geo-located datasets, e.g. remote sensing, where unlabeled data is often abundant but labeled data is scarce. We first show that due to their different characteristics, a non-trivial gap persists between contrastive and supervised learning on standard benchmarks. To close the gap, we propose novel training methods that exploit the spatio-temporal structure of remote sensing data. We leverage spatially aligned imag…
Graph Information Bottleneck for Remote Sensing Segmentation
School of computer and Information Engineering, Central South University of Forestry and Technology（计算机与信息工程学院，中央南大学林业科技学院）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
The existing SSCL of RSI is built based on constructing positive and negative sample pairs. However, due to the richness of RSI ground objects and the complexity of the RSI contextual semantics, the same RSI patches have the coexistence and imbalance of positive and negative samples, which causing the SSCL pushing negative samples far away while pushing positive samples far away, and vice versa. We call this the sample confounding issue (SCI). To solve this problem, we propose a False negAtive sampLes aware contraStive lEarning model (FALSE) for the semantic segmentation of high-resolution RSI…
SiamSeg: Self-Training with Contrastive Learning for Unsupervised Domain Adaptation Semantic Segmentation in Remote Sensing
College of Computer Science and Cyber Security, Chengdu University of Technology（成都理工大学计算机与网络安全学院）；College of Geophysics, Chengdu University of Technology（成都理工大学地球物理学院）；College of Mechanical and Vehicle Engineering, Changchun University（长春大学机械与车辆工程学院）
Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）
School of Information, University of California, Berkeley（加州大学伯克利分校信息学院）；IBM Chief Analytics Office（IBM首席分析办公室）
Climate change has caused reductions in river runoffs and aquifer recharge resulting in an increasingly unsustainable crop water demand from reduced freshwater availability. Achieving food security while deploying water in a sustainable manner will continue to be a major challenge necessitating careful monitoring and tracking of agricultural water usage. Historically, monitoring water usage has been a slow and expensive manual process with many imperfections and abuses. Ma-chine learning and remote sensing developments have increased the ability to automatically monitor irrigation patterns, bu…
Remote sensing change detection, identifying changes between scenes of the same location, is an active area of research with a broad range of applications. Recent advances in multimodal self-supervised pretraining have resulted in state-of-the-art methods which surpass vision models trained solely on optical imagery. In the remote sensing field, there is a wealth of overlapping 2D and 3D modalities which can be exploited to supervise representation learning in vision models. In this paper we propose Contrastive Surface-Image Pretraining (CSIP) for joint learning using optical RGB and above gro…
Center-guided Classifier for Semantic Segmentation of Remote Sensing Images
School of Software Technology, Zhejiang University（浙江大学软件学院）；Innovation Center of Yangtze River Delta, Zhejiang University（浙江大学长三角创新中心）；School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；City University of Hong Kong（香港城市大学）
Mitsubishi Electric Research Labs（三菱电机研究实验室）；Tufts University（塔夫茨大学）
In this paper, we study the problem of learning compact (low-dimensional) representations for sequential data that captures its implicit spatio-temporal cues. To maximize extraction of such informative cues from the data, we set the problem within the context of contrastive representation learning and to that end propose a novel objective via optimal transport. Specifically, our formulation seeks a low-dimensional subspace representation of the data that jointly (i) maximizes the distance of the data (embedded in this subspace) from an adversarial data distribution under the optimal transport,…
University of Pittsburgh（匹兹堡大学）；University of Notre Dame（圣母大学）
Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can help in this regard by learning a shared model while keeping training data local for privacy. Traditional FL requires fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to high labeling cost and the requirement of expertise. Contrastive learning (CL), as a self-supervised learning approach, can effectivel…
University of Pittsburgh（匹兹堡大学）；University of Notre Dame（圣母大学）；George Mason University（乔治梅森大学）
Supervised deep learning needs a large amount of labeled data to achieve high performance. However, in medical imaging analysis, each site may only have a limited amount of data and labels, which makes learning ineffective. Federated learning (FL) can learn a shared model from decentralized data. But traditional FL requires fully-labeled data for training, which is very expensive to obtain. Self-supervised contrastive learning (CL) can learn from unlabeled data for pre-training, followed by fine-tuning with limited annotations. However, when adopting CL in FL, the limited data diversity on eac…
University of Pittsburgh（匹兹堡大学）；George Mason University（乔治梅森大学）；University of Notre Dame（圣母大学）；Facebook（脸书）
Collaborative learning enables distributed clients to learn a shared model for prediction while keeping the training data local on each client. However, existing collaborative learning methods require fully-labeled data for training, which is inconvenient or sometimes infeasible to obtain due to the high labeling cost and the requirement of expertise. The lack of labels makes collaborative learning impractical in many realistic settings. Self-supervised learning can address this challenge by learning from unlabeled data. Contrastive learning (CL), a self-supervised learning approach, can effec…
From Pixels to Prose: Advancing Multi-Modal Language Models for Remote Sensing
Simon Fraser University（西蒙弗雷泽大学）；University of Minnesota - Twin Cities（明尼苏达大学双城分校）；Kyoto University（京都大学）；Georgia Institute of Technology（佐治亚理工学院）；National Taiwan Normal University（台湾师范大学）；Purdue University（普渡大学）；Emory University（埃默里大学）；The University of Texas at Dallas（德克萨斯大学达拉斯分校）
InfRS: Incremental Few-Shot Object Detection in Remote Sensing Images
School of Remote Sensing and Information Engineering, Wuhan University（武汉大学遥感信息工程学院）；Electronic Information School, Wuhan University（武汉大学电子信息学院）；Electrical and Computer Engineering, King Abdullah University of Science and Technology（阿卜杜拉国王科技大学电气与计算机工程学院）
DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning
ShanghaiTech University（上海科技大学）；National University of Defense Technology（国防科技大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Goldman Sachs（高盛）
Contributions to Label-Efficient Learning in Computer Vision and Remote Sensing
Université Bretagne Sud（南布列塔尼大学）；UMR IRISA（IRISA联合研究单位）；CentraleSupélec（中央理工-高等电力学院）；INRIA（法国国家信息与自动化研究所）；Technische Universität Berlin（柏林工业大学）；Université Savoie Mont Blanc（萨瓦勃朗峰大学）；Université de Caen Normandie（卡昂诺曼底大学）
Dynamic Dictionary Learning for Remote Sensing Image Segmentation
School of Computer Science and Technology, Beijing Jiaotong University（北京交通大学计算机科学与技术学院）；Department of Computer Technology and Applications, Qinghai University（青海大学计算机技术与应用系）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）
An Iterative Classification and Semantic Segmentation Network for Old Landslide Detection Using High-Resolution Remote Sensing Images
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Institute of Technology（北京理工大学）；China Aero Geophysical Survey and Remote Sensing Center for Natural Resources（中国自然资源航空物探遥感中心）；La Trobe University（乐卓博大学）
VFM-Guided Semi-Supervised Detection Transformer under Source-Free Constraints for Remote Sensing Object Detection
School of Information and Electronics, Beijing Institute of Technology（信息与电子学院，北京理工大学）；Beijing Institute of Technology Chongqing Innovation Center（北京理工大学重庆创新中心）；National Key Laboratory for Space-Born Intelligent Information Processing（空间智能信息处理国家重点实验室）
