Spatial--spectral FFPNet: Attention-Based Pyramid Network for Segmentation and Classification of Remote Sensing Images
Institute of Mountain Hazards and Environment, Chinese Academy of Sciences（中国科学院成都山地灾害与环境研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Bell Labs（贝尔实验室）；CAS Center for Excellence in Tibetan Plateau Earth Sciences, Chinese Academy of Sciences（中国科学院青藏高原地球科学卓越创新中心）；School of Economics and Management, Southwest Jiao Tong University（西南交通大学经济管理学院）
We consider the problem of segmentation and classification of high-resolution and hyperspectral remote sensing images. Unlike conventional natural (RGB) images, the inherent large scale and complex structures of remote sensing images pose major challenges such as spatial object distribution diversity and spectral information extraction when existing models are directly applied for image classification. In this study, we develop an attention-based pyramid network for segmentation and classification of remote sensing datasets. Attention mechanisms are used to develop the following modules: i) a novel and robust attention-based multi-scale fusion method effectively fuses useful spatial or spectral information at different and same scales; ii) a region pyramid attention mechanism using region-based attention addresses the target geometric size diversity in large-scale remote sensing images; and iii cross-scale attention} in our adaptive atrous spatial pyramid pooling network adapts to varied contents in a feature-embedded space. Different forms of feature fusion pyramid frameworks are established by combining these attention-based modules. First, a novel segmentation framework, called the heavy-weight spatial feature fusion pyramid network (FFPNet), is proposed to address the spatial problem of high-resolution remote sensing images. Second, an end-to-end spatial--spectral FFPNet is presented for classifying hyperspectral images. Experiments conducted on ISPRS Vaihingen and ISPRS Potsdam high-resolution datasets demonstrate the competitive segmentation accuracy achieved by the proposed heavy-weight spatial FFPNet. Furthermore, experiments on the Indian Pines and the University of Pavia hyperspectral datasets indicate that the proposed spatial--spectral FFPNet outperforms the current state-of-the-art methods in hyperspectral image classification.
Object counting, which aims to count the accurate number of object instances in images, has been attracting more and more attention. However, challenges such as large scale variation, complex background interference, and non-uniform density distribution greatly limit the counting accuracy, particularly striking in remote sensing imagery. To mitigate the above issues, this paper proposes a novel framework for dense object counting in remote sensing images, which incorporates a pyramidal scale module (PSM) and a global context module (GCM), dubbed PSGCNet, where PSM is used to adaptively capture…
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Estimating accurate number of interested objects from a given image is a challenging yet important task. Significant efforts have been made to address this problem and achieve great progress, yet counting number of ground objects from remote sensing images is barely studied. In this paper, we are interested in counting dense objects from remote sensing images. Compared with object counting in natural scene, this task is challenging in following factors: large scale variation, complex cluttered background and orientation arbitrariness. More importantly, the scarcity of data severely limits the …
Nanjing University of Information Science and Technology（南京信息工程大学）；Beijing Electro-Mechanical Engineering Institute（北京机电工程研究所）；University of Extremadura（埃斯特雷马杜拉大学）
Convolutional neural networks (CNNs) have attracted increasing attention in the remote sensing community. Most CNNs only take the last fully-connected layers as features for the classification of remotely sensed images, discarding the other convolutional layer features which may also be helpful for classification purposes. In this paper, we propose a new adaptive deep pyramid matching (ADPM) model that takes advantage of the features from all of the convolutional layers for remote sensing image classification. To this end, the optimal fusing weights for different convolutional layers are learn…
Beihang University（北京航空航天大学）；Hangzhou Innovation Institute, Beihang University（北京航空航天大学杭州创新研究院）
Object counting, whose aim is to estimate the number of objects from a given image, is an important and challenging computation task. Significant efforts have been devoted to addressing this problem and achieved great progress, yet counting the number of ground objects from remote sensing images is barely studied. In this paper, we are interested in counting dense objects from remote sensing images. Compared with object counting in a natural scene, this task is challenging in the following factors: large scale variation, complex cluttered background, and orientation arbitrariness. More importa…
TUBITAK Space Technologies Research Institute（土耳其科学技术研究理事会空间技术研究所）
Cloud detection from remotely observed data is a critical pre-processing step for various remote sensing applications. In particular, this problem becomes even harder for RGB color images, since there is no distinct spectral pattern for clouds, which is directly separable from the Earth surface. In this paper, we adapt a deep pyramid network (DPN) to tackle this problem. For this purpose, the network is enhanced with a pre-trained parameter model at the encoder layer. Moreover, the method is able to obtain accurate pixel-level segmentation and classification results from a set of noisy labeled…
Institute of Electronics, Chinese Academy of Sciences（中国科学院电子学研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Ship detection has been playing a significant role in the field of remote sensing for a long time but it is still full of challenges. The main limitations of traditional ship detection methods usually lie in the complexity of application scenarios, the difficulty of intensive object detection and the redundancy of detection region. In order to solve such problems above, we propose a framework called Rotation Dense Feature Pyramid Networks (R-DFPN) which can effectively detect ship in different scenes including ocean and port. Specifically, we put forward the Dense Feature Pyramid Network (DFPN…
Shanghai Jiao Tong University（上海交通大学）；GIPSA-lab, Université Grenoble Alpes, CNRS, Grenoble INP（格勒诺布尔阿尔卑斯大学、法国国家科学研究中心、格勒诺布尔国立理工学院联合GIPSA实验室）；University of Iceland（冰岛大学）；University of Leicester（莱斯特大学）
Detection of objects is extremely important in various aerial vision-based applications. Over the last few years, the methods based on convolution neural networks have made substantial progress. However, because of the large variety of object scales, densities, and arbitrary orientations, the current detectors struggle with the extraction of semantically strong features for small-scale objects by a predefined convolution kernel. To address this problem, we propose the rotation equivariant feature image pyramid network (REFIPN), an image pyramid network based on rotation equivariance convolutio…
Wuhan University（武汉大学）；University of Sydney（悉尼大学）；JD Explore Academy（京东探索研究院）
Road segmentation from remote sensing images is a challenging task with wide ranges of application potentials. Deep neural networks have advanced this field by leveraging the power of large-scale labeled data, which, however, are extremely expensive and time-consuming to acquire. One solution is to use cheap available data to train a model and deploy it to directly process the data from a specific application domain. Nevertheless, the well-known domain shift (DS) issue prevents the trained model from generalizing well on the target domain. In this paper, we propose a novel stagewise domain ada…
Shandong University（山东大学）；South China Normal University（华南师范大学）；China Telecom Corporation Limited（中国电信股份有限公司）；Nanjing University of Science and Technology（南京理工大学）；Tencent AI Lab（腾讯AI实验室）
City University of Hong Kong（香港城市大学）；Beijing Jiaotong University（北京交通大学）；City University of Hong Kong Shenzhen Research Institute（香港城市大学深圳研究院）；Tianjin University（天津大学）
Arising from the various object types and scales, diverse imaging orientations, and cluttered backgrounds in optical remote sensing image (RSI), it is difficult to directly extend the success of salient object detection for nature scene image to the optical RSI. In this paper, we propose an end-to-end deep network called LV-Net based on the shape of network architecture, which detects salient objects from optical RSIs in a purely data-driven fashion. The proposed LV-Net consists of two key modules, i.e., a two-stream pyramid module (L-shaped module) and an encoder-decoder module with nested co…
Shanghai Jiao Tong University（上海交通大学）；University of Leicester（莱斯特大学）；Central South University of Forestry and Technology（中南林业科技大学）；Massey University（梅西大学）
Road extraction in remote sensing images is of great importance for a wide range of applications. Because of the complex background, and high density, most of the existing methods fail to accurately extract a road network that appears correct and complete. Moreover, they suffer from either insufficient training data or high costs of manual annotation. To address these problems, we introduce a new model to apply structured domain adaption for synthetic image generation and road segmentation. We incorporate a feature pyramid network into generative adversarial networks to minimize the difference…
Shanghai Jiao Tong University（上海交通大学）；École de technologie supérieure, Université du Québec（魁北克大学高等技术学院）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；INRIA（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）
Small-object detection is a challenging problem. In the last few years, the convolution neural networks methods have been achieved considerable progress. However, the current detectors struggle with effective features extraction for small-scale objects. To address this challenge, we propose image pyramid single-shot detector (IPSSD). In IPSSD, single-shot detector is adopted combined with an image pyramid network to extract semantically strong features for generating candidate regions. The proposed network can enhance the small-scale features from a feature pyramid network. We evaluated the pe…
Object detection is a challenging task in remote sensing because objects only occupy a few pixels in the images, and the models are required to simultaneously learn object locations and detection. Even though the established approaches well perform for the objects of regular sizes, they achieve weak performance when analyzing small ones or getting stuck in the local minima (e.g. false object parts). Two possible issues stand in their way. First, the existing methods struggle to perform stably on the detection of small objects because of the complicated background. Second, most of the standard …
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology and Hydrology（英国生态与水文中心）
Semantic segmentation using fine-resolution remotely sensed images plays a critical role in many practical applications, such as urban planning, environmental protection, natural and anthropogenic landscape monitoring, etc. However, the automation of semantic segmentation, i.e., automatic categorization/labeling and segmentation is still a challenging task, particularly for fine-resolution images with huge spatial and spectral complexity. Addressing such a problem represents an exciting research field, which paves the way for scene-level landscape pattern analysis and decision making. In this …
Centre for Research-based Innovation Visual Intelligence（可视化研究型创新中心）
Multi-modality data is becoming readily available in remote sensing (RS) and can provide complementary information about the Earth's surface. Effective fusion of multi-modal information is thus important for various applications in RS, but also very challenging due to large domain differences, noise, and redundancies. There is a lack of effective and scalable fusion techniques for bridging multiple modality encoders and fully exploiting complementary information. To this end, we propose a new multi-modality network (MultiModNet) for land cover mapping of multi-modal remote sensing data based o…
Northeastern University（东北大学）；Shengjing Hospital, China Medical University（中国医科大学附属盛京医院）
In this paper, we propose a dual aggregation network to adaptively aggregate different information in infant brain MRI segmentation. More precisely, we added two modules based on 3D-UNet to better model information at different levels and locations. The dilated convolution pyramid downsampling module is mainly to solve the problem of loss of spatial information on the downsampling process, and it can effectively save details while reducing the resolution. The self-attention module can integrate the remote dependence on the feature maps in two dimensions of spatial and channel, effectively impr…
City University of Hong Kong（香港城市大学）；Beijing Jiaotong University（北京交通大学）
Despite the remarkable advances in visual saliency analysis for natural scene images (NSIs), salient object detection (SOD) for optical remote sensing images (RSIs) still remains an open and challenging problem. In this paper, we propose an end-to-end Dense Attention Fluid Network (DAFNet) for SOD in optical RSIs. A Global Context-aware Attention (GCA) module is proposed to adaptively capture long-range semantic context relationships, and is further embedded in a Dense Attention Fluid (DAF) structure that enables shallow attention cues flow into deep layers to guide the generation of high-leve…
Building extraction from aerial images has several applications in problems such as urban planning, change detection, and disaster management. With the increasing availability of data, Convolutional Neural Networks (CNNs) for semantic segmentation of remote sensing imagery has improved significantly in recent years. However, convolutions operate in local neighborhoods and fail to capture non-local features that are essential in semantic understanding of aerial images. In this work, we propose to improve building segmentation of different sizes by capturing long-range dependencies using context…
Recently, change detection (CD) of remote sensing images have achieved great progress with the advances of deep learning. However, current methods generally deliver incomplete CD regions and irregular CD boundaries due to the limited representation ability of the extracted visual features. To relieve these issues, in this work we propose a novel learning framework named Fully Transformer Network (FTN) for remote sensing image CD, which improves the feature extraction from a global view and combines multi-level visual features in a pyramid manner. More specifically, the proposed framework first…
A Multitask Deep Learning Model for Classification and Regression of Hyperspectral Images: Application to the large-scale dataset
Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay（印度理工学院孟买分校资源工程研究中心）；LISTIC, Université Savoie Mont-Blanc（萨瓦勃朗峰大学LISTIC实验室）；Indian Institute of Remote Sensing, ISRO（印度空间研究组织印度遥感研究所）
Remote sensor image object detection is an important technology for Earth observation, and is used in various tasks such as forest fire monitoring and ocean monitoring. Image object detection technology, despite the significant developments, is struggling to handle remote sensor images and small-scale objects, due to the limited pixels of small objects. Numerous existing studies have demonstrated that an effective way to promote small object detection is to introduce the spatial context. Meanwhile, recent researches for image classification have shown that spectral convolution operations can p…
LOANet: A Lightweight Network Using Object Attention for Extracting Buildings and Roads from UAV Aerial Remote Sensing Images
School of Medical Instruments, Shanghai University of Medicine and Health Sciences（上海健康医学院医疗器械学院）；School of Health Sciences and Engineering, University of Shanghai for Science and Technology（上海理工大学健康科学与工程学院）；Shanghai Children’s Medical Center, School of Medicine, Shanghai Jiao Tong University（上海交通大学医学院附属上海儿童医学中心）；School of Communication & Electronic Engineering, East China Normal University（华东师范大学通信与电子工程学院）；Institute of Seismology, China Earthquake Administration（中国地震局地震研究所）
Learning to Aggregate Multi-Scale Context for Instance Segmentation in Remote Sensing Images
The Hong Kong Polytechnic University（香港理工大学）；Wuhan University（武汉大学）；Changjiang Spatial Information Technology Engineering Co., Ltd.（长江空间信息技术工程有限公司）；Peng Cheng Laboratory（鹏城实验室）
Beihang University（北京航空航天大学）；University of Central Florida（中佛罗里达大学）；Lancaster University（兰卡斯特大学）；Huawei Technologies Co. Ltd.（华为技术有限公司）
Compression artifacts reduction (CAR) is a challenging problem in the field of remote sensing. Most recent deep learning based methods have demonstrated superior performance over the previous hand-crafted methods. In this paper, we propose an end-to-end one-two-one (OTO) network, to combine different deep models, i.e., summation and difference models, to solve the CAR problem. Particularly, the difference model motivated by the Laplacian pyramid is designed to obtain the high frequency information, while the summation model aggregates the low frequency information. We provide an in-depth inves…
Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost, thereby providing a feasible way to acquire high-resolution remote sensing images, which are difficult to obtain due to the high cost of acquisition equipment and complex weather. Clearly, image super-resolution is a severe ill-posed problem. Fortunately, with the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. In this paper, we propose a network based on the generative adversarial network (GA…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Leicester（莱斯特大学）
Due to the more dramatic multi-scale variations and more complicated foregrounds and backgrounds in optical remote sensing images (RSIs), the salient object detection (SOD) for optical RSIs becomes a huge challenge. However, different from natural scene images (NSIs), the discussion on the optical RSI SOD task still remains scarce. In this paper, we propose a multi-scale context network, namely MSCNet, for SOD in optical RSIs. Specifically, a multi-scale context extraction module is adopted to address the scale variation of salient objects by effectively learning multi-scale contextual informa…
DSFC-Net: A Dual-Encoder Spatial and Frequency Co-Awareness Network for Rural Road Extraction
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所多模态人工智能系统国家重点实验室）；Department of Earth System Science, Tsinghua University（清华大学地球系统科学系）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Information and Electrical Engineering, China Agricultural University（中国农业大学信息与电气工程学院）
Beijing Institute of Technology（北京理工大学）；German Aerospace Center (DLR)（德国航空航天中心）；Technical University of Munich (TUM)（慕尼黑工业大学）；Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔国立高等物理、数学与信息学院）；University of Iceland（冰岛大学）
With the rapid development of spaceborne imaging techniques, object detection in optical remote sensing imagery has drawn much attention in recent decades. While many advanced works have been developed with powerful learning algorithms, the incomplete feature representation still cannot meet the demand for effectively and efficiently handling image deformations, particularly objective scaling and rotation. To this end, we propose a novel object detection framework, called optical remote sensing imagery detector (ORSIm detector), integrating diverse channel features extraction, feature learning…
Deep learning (DL) stereo matching methods gained great attention in remote sensing satellite datasets. However, most of these existing studies conclude assessments based only on a few/single stereo images lacking a systematic evaluation on how robust DL methods are on satellite stereo images with varying radiometric and geometric configurations. This paper provides an evaluation of four DL stereo matching methods through hundreds of multi-date multi-site satellite stereo pairs with varying geometric configurations, against the traditional well-practiced Census-SGM (Semi-global matching), to c…
German Aerospace Center（德国航空航天中心）；Technical University of Munich（慕尼黑工业大学）
Automatic multi-class object detection in remote sensing images in unconstrained scenarios is of high interest for several applications including traffic monitoring and disaster management. The huge variation in object scale, orientation, category, and complex backgrounds, as well as the different camera sensors pose great challenges for current algorithms. In this work, we propose a new method consisting of a novel joint image cascade and feature pyramid network with multi-size convolution kernels to extract multi-scale strong and weak semantic features. These features are fed into rotation-b…
ChangeDINO: DINOv3-Driven Building Change Detection in Optical Remote Sensing Imagery
1 Inst.\ of Data Science, National Cheng Kung University, \ , Taiwan - 2 Inst.\ of Intelligent Systems College of Artificial Intelligence, \ Yang Ming Chiao Tung University, Hsinchu, Taiwan
SemDINO: Foundation Prior-Guided Cross-Temporal Semantic Alignment Network for Remote Sensing Change Detection
Xinjiang Institute of Ecology and Geography, Chinese Academy of Sciences（中国科学院新疆生态与地理研究所）；University of Chinese Academy of Sciences（中国科学院大学）；School of Computer Science, Xiangtan University（湘潭大学计算机科学学院）；College of Information and Communication Engineering, Harbin Engineering University（哈尔滨工程大学信息与通信工程学院）
Generalization-Enhanced Few-Shot Object Detection in Remote Sensing
China Academy of Electronics and Information Technology（中国电子科技集团公司电子科学研究院）；National Satellite Meteorological Center（国家卫星气象中心）；Innovation Center for FengYun Meteorological Satellite（风云气象卫星创新中心）；China Meteorological Administration（中国气象局）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；School of Automation and Electrical Engineering, University of Science and Technology Beijing（北京科技大学自动化与电气工程学院）；New York University Abu Dhabi（纽约大学阿布扎比分校）
A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation
School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳）科学与工程学院）；School of Information Science and Engineering, Wuhan University of Science and Technology（武汉科技大学信息科学与工程学院）；Department of Geography, The University of Hong Kong（香港大学地理系）
Threshold Attention Network for Semantic Segmentation of Remote Sensing Images
State Key Laboratory of Public Big Data, College of Computer Science and Technology, Guizhou University（贵州大学计算机科学与技术学院 公共大数据国家重点实验室）；School of Mathematics and Big Data, Guizhou Education University（贵州师范学院数学与大数据学院）
Southwest Jiaotong University（西南交通大学）；Central South University（中南大学）
Accurately and efficiently extracting building footprints from a wide range of remote sensed imagery remains a challenge due to their complex structure, variety of scales and diverse appearances. Existing convolutional neural network (CNN)-based building extraction methods are complained that they cannot detect the tiny buildings because the spatial information of CNN feature maps are lost during repeated pooling operations of the CNN, and the large buildings still have inaccurate segmentation edges. Moreover, features extracted by a CNN are always partial which restricted by the size of the r…
Adaptive Multi-Scale Correlation Meta-Network for Few-Shot Remote Sensing Image Classification
School of Computer Science, UPES, Dehradun, 248007, Uttarakhand, India.（UPES计算机科学学院）；School of Computer Science and Engineering, Galgotias University, India.（加尔各答大学计算机科学与工程学院）；Computer Science and Information Systems, BITS Pilani, Pilani Rajasthan, India.（比什努学院计算机科学与信息系统）
DFPENet-geology: A Deep Learning Framework for High Precision Recognition and Segmentation of Co-seismic Landslides
Institute of Mountain Hazards and Environment, Chinese Academy of Sciences（中国科学院成都山地灾害与环境研究所）；CAS Center for Excellence in Tibetan Plateau Earth Sciences, Chinese Academy of Sciences（中国科学院青藏高原地球科学卓越创新中心）；University of Chinese Academy of Sciences（中国科学院大学）；Chengdu University of Technology（成都理工大学）；Sichuan Geomatics Center（四川省测绘地理信息中心）
