Detecting topographic changes in the urban environment has always been an important task for urban planning and monitoring. In practice, remote sensing data are often available in different modalities and at different time epochs. Change detection between multimodal data can be very challenging since the data show different characteristics. Given 3D laser scanning point clouds and 2D imagery from different epochs, this paper presents a framework to detect building and tree changes. First, the 2D and 3D data are transformed to image patches, respectively. A Siamese CNN is then employed to detect candidate changes between the two epochs. Finally, the candidate patch-based changes are grouped and verified as individual object changes. Experiments on the urban data show that 86.4\% of patch pairs can be correctly classified by the model.
University of Electronic Science and Technology of China（电子科技大学）；Truly Opto-Electronics Co., Ltd.（信利光电股份有限公司）
The Siamese network is becoming the mainstream in change detection of remote sensing images (RSI). However, in recent years, the development of more complicated structure, module and training processe has resulted in the cumbersome model, which hampers their application in large-scale RSI processing. To this end, this paper proposes an extremely lightweight Siamese network (LSNet) for RSI change detection, which replaces standard convolution with depthwise separable atrous convolution, and removes redundant dense connections, retaining only valid feature flows while performing Siamese feature …
Siamese networks are widely used for remote sensing change detection tasks. A vanilla siamese network has two identical feature extraction branches which share weights, these two branches work independently and the feature maps are not fused until about to be sent to a decoder head. However we find that it is critical to exchange information between two feature extraction branches at early stage for change detection task. In this work we present Mutual-Attention Siamese Network (MASNet), a general siamese network with mutual-attention plug-in, so to exchange information between the two feature…
In recent years, building change detection methods have made great progress by introducing deep learning, but they still suffer from the problem of the extracted features not being discriminative enough, resulting in incomplete regions and irregular boundaries. To tackle this problem, we propose a dual task constrained deep Siamese convolutional network (DTCDSCN) model, which contains three sub-networks: a change detection network and two semantic segmentation networks. DTCDSCN can accomplish both change detection and semantic segmentation at the same time, which can help to learn more discrim…
This paper presents a transformer-based Siamese network architecture (abbreviated by ChangeFormer) for Change Detection (CD) from a pair of co-registered remote sensing images. Different from recent CD frameworks, which are based on fully convolutional networks (ConvNets), the proposed method unifies hierarchically structured transformer encoder with Multi-Layer Perception (MLP) decoder in a Siamese network architecture to efficiently render multi-scale long-range details required for accurate CD. Experiments on two CD datasets show that the proposed end-to-end trainable ChangeFormer architect…
We tackle the problem of one-shot instance segmentation: Given an example image of a novel, previously unknown object category, find and segment all objects of this category within a complex scene. To address this challenging new task, we propose Siamese Mask R-CNN. It extends Mask R-CNN by a Siamese backbone encoding both reference image and scene, allowing it to target detection and segmentation towards the reference category. We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object c…
Ethiopian Institute of Technology - Mekelle（埃塞俄比亚理工学院 - 梅克莱）；Institute of Geo-Information and Earth Observation Science（地理信息与地球观测科学研究所）
Change detection is a fast-growing discipline in the areas of computer vision and remote sensing. In this work, we designed and developed a variant of convolutional neural network (CNN), known as Siamese CNN to extract features from pairs of Sentinel-2 temporal images of Mekelle city captured at different times and detect changes due to urbanization: buildings and roads. The detection capability of the proposed was measured in terms of overall accuracy (95.8), Kappa measure (72.5), recall (76.5), precision (77.7), F1 measure (77.1). The model has achieved a good performance in terms of most of…
SiamixFormer: a fully-transformer Siamese network with temporal Fusion for accurate building detection and change detection in bi-temporal remote sensing images
Nanjing University of Science and Technology（南京理工大学）
Change detection (CD) of remote sensing images is to detect the change region by analyzing the difference between two bitemporal images. It is extensively used in land resource planning, natural hazards monitoring and other fields. In our study, we propose a novel Siamese neural network for change detection task, namely Dual-UNet. In contrast to previous individually encoded the bitemporal images, we design an encoder differential-attention module to focus on the spatial difference relationships of pixels. In order to improve the generalization of networks, it computes the attention weights be…
Central South University（中南大学）；National University of Defense Technology（中国人民解放军国防科技大学）
Change detection is a basic task of remote sensing image processing. The research objective is to identity the change information of interest and filter out the irrelevant change information as interference factors. Recently, the rise of deep learning has provided new tools for change detection, which have yielded impressive results. However, the available methods focus mainly on the difference information between multitemporal remote sensing images and lack robustness to pseudo-change information. To overcome the lack of resistance of current methods to pseudo-changes, in this paper, we propo…
Hochschule Bonn-Rhein-Sieg（波恩-莱茵-锡格应用技术大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Application of underwater robots are on the rise, most of them are dependent on sonar for underwater vision, but the lack of strong perception capabilities limits them in this task. An important issue in sonar perception is matching image patches, which can enable other techniques like localization, change detection, and mapping. There is a rich literature for this problem in color images, but for acoustic images, it is lacking, due to the physics that produce these images. In this paper we improve on our previous results for this problem (Valdenegro-Toro et al, 2017), instead of modeling feat…
Change detection (CD) is an important problem in remote sensing, especially in disaster time for urban management. Most existing traditional methods for change detection are categorized based on pixel or objects. Object-based models are preferred to pixel-based methods for handling very high-resolution remote sensing (VHR RS) images. Such methods can benefit from the ongoing research on deep learning. In this paper, a fully automatic change-detection algorithm on VHR RS images is proposed that deploys Fully Convolutional Siamese Concatenate networks (FC-Siam-Conc). The proposed method uses pre…
RemoteDet-Mamba: A Hybrid Mamba-CNN Network for Multi-modal Object Detection in Remote Sensing Images
School of Computer Science, Beijing University of Posts and Telecommunications, Beijing, China（计算机科学学院，北京邮电大学）；School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, China（电子工程学院，北京邮电大学）
In recent years, Fully Convolutional Networks (FCN) has been widely used in various semantic segmentation tasks, including multi-modal remote sensing imagery. How to fuse multi-modal data to improve the segmentation performance has always been a research hotspot. In this paper, a novel end-toend fully convolutional neural network is proposed for semantic segmentation of natural color, infrared imagery and Digital Surface Models (DSM). It is based on a modified DeepUNet and perform the segmentation in a multi-task way. The channels are clustered into groups and processed on different task pipel…
MO R-CNN: Multispectral Oriented R-CNN for Object Detection in Remote Sensing Image
College of Electronics and Information Engineering, Sichuan University（四川大学电子信息工程学院）；School of Mechanical Engineering and Automation, Fuzhou University（福州大学机械工程与自动化学院）；Yunnan Key Laboratory of Software Engineering, Yunnan University（云南软件工程重点实验室）
针对多光谱定向目标检测中模态差异与计算开销问题，提出轻量级框架 MO R-CNN，通过异构特征提取、单模态监督和条件标签融合提升检测性能。
CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；School of Aeronautics and Astronautics, Zhejiang University（浙江大学航空航天学院）；College of Computer Science, Zhejiang University（浙江大学计算机学院）
Change detection (CD) is one of the most vital applications in remote sensing. Recently, deep learning has achieved promising performance in the CD task. However, the deep models are task-specific and CD data set bias often exists, hence it is inevitable that deep CD models would suffer degraded performance after transferring it from original CD data set to new ones, making manually label numerous samples in the new data set unavoidable, which costs a large amount of time and human labor. How to learn a transferable CD model in the data set with enough labeled data (original domain) but can we…
Recently, the convolutional neural network has brought impressive improvements for object detection. However, detecting tiny objects in large-scale remote sensing images still remains challenging. First, the extreme large input size makes the existing object detection solutions too slow for practical use. Second, the massive and complex backgrounds cause serious false alarms. Moreover, the ultratiny objects increase the difficulty of accurate detection. To tackle these problems, we propose a unified and self-reinforced network called remote sensing region-based convolutional neural network ($\…
In this work we consider the application of convolutional neural networks (CNNs) for pixel-wise labeling (a.k.a., semantic segmentation) of remote sensing imagery (e.g., aerial color or hyperspectral imagery). Remote sensing imagery is usually stored in the form of very large images, referred to as "tiles", which are too large to be segmented directly using most CNNs and their associated hardware. As a result, during label inference, smaller sub-images, called "patches", are processed individually and then "stitched" (concatenated) back together to create a tile-sized label map. This approach …
With the advancement of remote-sensed imaging large volumes of very high resolution land cover images can now be obtained. Automation of object recognition in these 2D images, however, is still a key issue. High intra-class variance and low inter-class variance in Very High Resolution (VHR) images hamper the accuracy of prediction in object recognition tasks. Most successful techniques in various computer vision tasks recently are based on deep supervised learning. In this work, a deep Convolutional Neural Network (CNN) based on symmetric encoder-decoder architecture with skip connections is e…
Deep learning based change detection methods have received wide attentoion, thanks to their strong capability in obtaining rich features from images. However, existing AI-based CD methods largely rely on three functionality-enhancing modules, i.e., semantic enhancement, attention mechanisms, and correspondence enhancement. The stacking of these modules leads to great model complexity. To unify these three modules into a simple pipeline, we introduce Relational Change Detection Transformer (RCDT), a novel and simple framework for remote sensing change detection tasks. The proposed RCDT consists…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Today deep convolutional neural networks (CNNs) push the limits for most computer vision problems, define trends, and set state-of-the-art results. In remote sensing tasks such as object detection and semantic segmentation, CNNs reach the SotA performance. However, for precise performance, CNNs require much high-quality training data. Rare objects and the variability of environmental conditions strongly affect prediction stability and accuracy. To overcome these data restrictions, it is common to consider various approaches including data augmentation techniques. This study focuses on the deve…
School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Instrumentation Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学仪器科学与工程学院）；Harbin Institute of Technology Suzhou Research Institute（哈尔滨工业大学苏州研究院）；Ningbo Haichuang Group Co., Ltd.（宁波海创集团有限公司）；School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Information Engineering, North China University of Water Resources and Electric Power（中国北方水利电力大学信息工程学院）；Department of Geo-spatial Information, Information Engineering University（信息工程大学地理信息学院）；Innovation Center of Yangtze River Delta, Zhejiang University（长三角创新中心，浙江大学）
Semantic segmentation of remotely sensed images plays a crucial role in precision agriculture, environmental protection, and economic assessment. In recent years, substantial fine-resolution remote sensing images are available for semantic segmentation. However, due to the complicated information caused by the increased spatial resolution, state-of-the-art deep learning algorithms normally utilize complex network architectures for segmentation, which usually incurs high computational complexity. Specifically, the high-caliber performance of the convolutional neural network (CNN) heavily relies…
KTH Royal Inst. of Technology（KTH皇家理工学院）；University of Rwanda（卢旺达大学）
To better understand current trends of urban population growth in Sub-Saharan Africa, high-quality spatiotemporal population estimates are necessary. While the joint use of remote sensing and deep learning has achieved promising results for population distribution estimation, most of the current work focuses on fine-scale spatial predictions derived from single date census, thereby neglecting temporal analyses. In this work, we focus on evaluating how deep learning change detection techniques can unravel temporal population dynamics at short intervals. Since Post-Classification Comparison (PCC…
École de technologie supérieure（高等技术学院）；Genetec Inc.（Genetec公司）
Visual object tracking is an important function in many real-time video surveillance applications, such as localization and spatio-temporal recognition of persons. In real-world applications, an object detector and tracker must interact on a periodic basis to discover new objects, and thereby to initiate tracks. Periodic interactions with the detector can also allow the tracker to validate and/or update its object template with new bounding boxes. However, bounding boxes provided by a state-of-the-art detector are noisy, due to changes in appearance, background and occlusion, which can cause t…
Deep neural networks have been investigated in learning latent representations of medical images, yet most of the studies limit their approach in a single supervised convolutional neural network (CNN), which usually rely heavily on a large scale annotated dataset for training. To learn image representations with less supervision involved, we propose a deep Siamese CNN (SCNN) architecture that can be trained with only binary image pair information. We evaluated the learned image representations on a task of content-based medical image retrieval using a publicly available multiclass diabetic ret…
In this paper, we deal with the problem of object detection on remote sensing images. Previous methods have developed numerous deep CNN-based methods for object detection on remote sensing images and the report remarkable achievements in detection performance and efficiency. However, current CNN-based methods mostly require a large number of annotated samples to train deep neural networks and tend to have limited generalization abilities for unseen object categories. In this paper, we introduce a few-shot learning-based method for object detection on remote sensing images where only a few anno…
Vision Mamba in Remote Sensing: A Comprehensive Survey of Techniques, Applications and Outlook
School of Advanced Technology, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学先进技术学院）；School of Electronics and Information Engineering, Beihang University（北航电子信息工程学院）；Department of Paediatrics, Cambridge University（剑桥大学儿科部）；School of Computing and Mathematical Sciences, University of Leicester（莱斯特大学计算与数学科学学院）；Department of Computing Science, Robert Gordon University（罗伯特·戈登大学计算机科学系）；National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家实验室）；Nanyang Technological University, Singapore（新加坡南洋理工大学）；Department of Computer Science, University of Liverpool（利物浦大学计算机科学系）
Benefiting from the great success of deep learning in computer vision, CNN-based object detection methods have drawn significant attentions. Various frameworks have been proposed which show awesome and robust performance for a large range of datasets. However, for building detection in remote sensing images, buildings always pose a diversity of orientations which makes it a challenge for the application of off-the-shelf methods to building detection. In this work, we aim to integrate orientation regression into the popular axis-aligned bounding-box detection method to tackle this problem. To a…
STeInFormer: Spatial-Temporal Interaction Transformer Architecture for Remote Sensing Change Detection
School of Software Technology, Zhejiang University（浙江大学软件学院）；School of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Earth Sciences, Zhejiang University（地球科学学院）；Innovation Center of Yangtze River Delta, Zhejiang University（浙江大学长三角创新中心）
ChangeDINO: DINOv3-Driven Building Change Detection in Optical Remote Sensing Imagery
1 Inst.\ of Data Science, National Cheng Kung University, \ , Taiwan - 2 Inst.\ of Intelligent Systems College of Artificial Intelligence, \ Yang Ming Chiao Tung University, Hsinchu, Taiwan
Object detection is a fundamental and challenging problem in aerial and satellite image analysis. More recently, a two-stage detector Faster R-CNN is proposed and demonstrated to be a promising tool for object detection in optical remote sensing images, while the sparse and dense characteristic of objects in remote sensing images is complexity. It is unreasonable to treat all images with the same region proposal strategy, and this treatment limits the performance of two-stage detectors. In this paper, we propose a novel and effective approach, named deep adaptive proposal network (DAPNet), add…
Seed Feature Maps-based CNN Models for LEO Satellite Remote Sensing Services
Sun Yat-sen University（中山大学）；Beijing Jiaotong University（北京交通大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Southern University of Science and Technology（南方科技大学）；New York University（纽约大学）；Michigan State University（密歇根州立大学）
This paper addresses the problem of large scale image retrieval, with the aim of accurately ranking the similarity of a large number of images to a given query image. To achieve this, we propose a novel Siamese network. This network consists of two computational strands, each comprising of a CNN component followed by a Fisher vector component. The CNN component produces dense, deep convolutional descriptors that are then aggregated by the Fisher Vector method. Crucially, we propose to simultaneously learn both the CNN filter weights and Fisher Vector model parameters. This allows us to account…
Multispectral disparity estimation is a difficult task for many reasons: it has all the same challenges as traditional visible-visible disparity estimation (occlusions, repetitive patterns, textureless surfaces), in addition of having very few common visual information between images (e.g. color information vs. thermal information). In this paper, we propose a new CNN architecture able to do disparity estimation between images from different spectrum, namely thermal and visible in our case. Our proposed model takes two patches as input and proceeds to do domain feature extraction for each of t…
University of Helsinki（赫尔辛基大学）；Finnish Environment Institute（芬兰环境研究所）
Semantic segmentation by convolutional neural networks (CNN) has advanced the state of the art in pixel-level classification of remote sensing images. However, processing large images typically requires analyzing the image in small patches, and hence features that have large spatial extent still cause challenges in tasks such as cloud masking. To support a wider scale of spatial features while simultaneously reducing computational requirements for large satellite images, we propose an architecture of two cascaded CNN model components successively processing undersampled and full resolution ima…
National University of Defense Technology（国防科技大学）；Hunan Univeristy（湖南大学）
Recently, the majority of visual trackers adopt Convolutional Neural Network (CNN) as their backbone to achieve high tracking accuracy. However, less attention has been paid to the potential adversarial threats brought by CNN, including Siamese network. In this paper, we first analyze the existing vulnerabilities in Siamese trackers and propose the requirements for a successful adversarial attack. On this basis, we formulate the adversarial generation problem and propose an end-to-end pipeline to generate a perturbed texture map for the 3D object that causes the trackers to fail. Finally, we c…
Amazon Development Centre (India) Pvt. Ltd.（亚马逊（印度）开发中心私人有限公司）
Image similarity involves fetching similar looking images given a reference image. Our solution called SimNet, is a deep siamese network which is trained on pairs of positive and negative images using a novel online pair mining strategy inspired by Curriculum learning. We also created a multi-scale CNN, where the final image embedding is a joint representation of top as well as lower layer embedding's. We go on to show that this multi-scale siamese network is better at capturing fine grained image similarities than traditional CNN's.
