Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
Gong Cheng, Xingxing Xie, Junwei Han, Lei Guo, Gui-Song Xia
备注：
This manuscript is the accepted version for IEEE JSTARS
发表信息：
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 13: 3735-3756, 2020
Remote sensing image scene classification, which aims at labeling remote sensing images with a set of semantic categories based on their contents, has broad applications in a range of fields. Propelled by the powerful feature learning capabilities of deep neural networks, remote sensing image scene classification driven by deep learning has drawn remarkable attention and achieved significant breakthroughs. However, to the best of our knowledge, a comprehensive review of recent achievements regarding deep learning for scene classification of remote sensing images is still lacking. Considering the rapid evolution of this field, this paper provides a systematic survey of deep learning methods for remote sensing image scene classification by covering more than 160 papers. To be specific, we discuss the main challenges of remote sensing image scene classification and survey (1) Autoencoder-based remote sensing image scene classification methods, (2) Convolutional Neural Network-based remote sensing image scene classification methods, and (3) Generative Adversarial Network-based remote sensing image scene classification methods. In addition, we introduce the benchmarks used for remote sensing image scene classification and summarize the performance of more than two dozen of representative algorithms on three commonly-used benchmark data sets. Finally, we discuss the promising opportunities for further research.
A Decade of Deep Learning for Remote Sensing Spatiotemporal Fusion: Advances, Challenges, and Opportunities
School of Computer Science, China University of Geosciences (Wuhan)（中国地质大学（武汉）计算机科学学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空航天信息研究所）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
Diffusion Models Meet Remote Sensing: Principles, Methods, and Perspectives
College of Electrical and Information Engineering, Hunan University（湖南大学电气与信息工程学院）；School of Automation, Central South University（中南大学自动化学院）；Changsha University of Science and Technology（长沙理工大学）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Lancaster University（兰卡斯特大学）；Xidian University（西安电子科技大学）
This paper provides an overview of how recent advances in machine learning and the availability of data from earth observing satellites can dramatically improve our ability to automatically map croplands over long period and over large regions. It discusses three applications in the domain of crop monitoring where ML approaches are beginning to show great promise. For each application, it highlights machine learning challenges, proposed approaches, and recent results. The paper concludes with discussion of major challenges that need to be addressed before ML approaches will reach their full po…
University of Campinas（坎皮纳斯州立大学）；Wageningen University & Research（瓦赫宁根大学及研究中心）
OpenStreetMap (OSM) is a community-based, freely available, editable map service that was created as an alternative to authoritative ones. Given that it is edited mainly by volunteers with different mapping skills, the completeness and quality of its annotations are heterogeneous across different geographical locations. Despite that, OSM has been widely used in several applications in {Geosciences}, Earth Observation and environmental sciences. In this work, we present a review of recent methods based on machine learning to improve and use OSM data. Such methods aim either 1) at improving the …
Scene classification is a fundamental task in interpretation of remote sensing images, and has become an active research topic in remote sensing community due to its important role in a wide range of applications. Over the past years, tremendous efforts have been made for developing powerful approaches for scene classification of remote sensing images, evolving from the traditional bag-of-visual-words model to the new generation deep convolutional neural networks (CNNs). The deep CNN based methods have exhibited remarkable breakthrough on performance, dramatically outperforming previous method…
School of Computer and Artificial Intelligence, Zhengzhou University（计算机与人工智能学院，郑州大学）；Wuhan University（武汉大学）；Harbin Institution of Technology（哈尔滨理工大学）；Dalian Maritime University（大连海事大学）；National Tsinghua University（清华大学）
Shen Yuan Honors College, Beihang University（北航沈元荣誉学院）；Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University（北京航空航天大学宇航学院航天智能科学与技术系）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology（中国空间技术研究院钱学森空间技术实验室）
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））
Fully automatic large-scale land cover mapping belongs to the core challenges addressed by the remote sensing community. Usually, the basis of this task is formed by (supervised) machine learning models. However, in spite of recent growth in the availability of satellite observations, accurate training data remains comparably scarce. On the other hand, numerous global land cover products exist and can be accessed often free-of-charge. Unfortunately, these maps are typically of a much lower resolution than modern day satellite imagery. Besides, they always come with a significant amount of nois…
Univ. Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；GIPSA-lab（GIPSA实验室）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；the University of Tokyo（东京大学）；RIKEN Center for Advanced Intelligence Project (AIP), RIKEN（理化学研究所革新智能综合研究中心）；Xi’an Jiaotong University（西安交通大学）；INRIA（法国国家信息与自动化研究所）；LJK（让·昆兹曼实验室）；Mississippi State University（密西西比州立大学）
Classification and identification of the materials lying over or beneath the Earth's surface have long been a fundamental but challenging research topic in geoscience and remote sensing (RS) and have garnered a growing concern owing to the recent advancements of deep learning techniques. Although deep networks have been successfully applied in single-modality-dominated classification tasks, yet their performance inevitably meets the bottleneck in complex scenes that need to be finely classified, due to the limitation of information diversity. In this work, we provide a baseline solution to the…
Panoptic segmentation combines instance and semantic predictions, allowing the detection of "things" and "stuff" simultaneously. Effectively approaching panoptic segmentation in remotely sensed data can be auspicious in many challenging problems since it allows continuous mapping and specific target counting. Several difficulties have prevented the growth of this task in remote sensing: (a) most algorithms are designed for traditional images, (b) image labelling must encompass "things" and "stuff" classes, and (c) the annotation format is complex. Thus, aiming to solve and increase the operabi…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所）；Univ. Grenoble Alpes（格勒诺布尔大学）；CNRS（法国国家科学研究中心）；Grenoble INP（格勒诺布尔理工学院）；RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所智能情报研究中心）；Graduate School of Frontier Sciences, the University of Tokyo（东京大学前沿科学研究生院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Wuhan University（武汉大学）；INRIA（法国国家信息与自动化研究所）；LJK（格勒诺布尔大学-法国国家科学研究中心-格勒诺布尔理工学院联合实验室）
Hyperspectral imaging, also known as image spectrometry, is a landmark technique in geoscience and remote sensing (RS). In the past decade, enormous efforts have been made to process and analyze these hyperspectral (HS) products mainly by means of seasoned experts. However, with the ever-growing volume of data, the bulk of costs in manpower and material resources poses new challenges on reducing the burden of manual labor and improving efficiency. For this reason, it is, therefore, urgent to develop more intelligent and automatic approaches for various HS RS applications. Machine learning (ML)…
Central South University（中南大学）；Hefei Institutes of Physical Science, Chinese Academy of Sciences（中国科学院合肥物质科学研究院）
Remote sensing image scene classification is a fundamental but challenging task in understanding remote sensing images. Recently, deep learning-based methods, especially convolutional neural network-based (CNN-based) methods have shown enormous potential to understand remote sensing images. CNN-based methods meet with success by utilizing features learned from data rather than features designed manually. The feature-learning procedure of CNN largely depends on the architecture of CNN. However, most of the architectures of CNN used for remote sensing scene classification are still designed by h…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
Data-driven methods such as convolutional neural networks (CNNs) are known to deliver state-of-the-art performance on image recognition tasks when the training data are abundant. However, in some instances, such as change detection in remote sensing images, annotated data cannot be obtained in sufficient quantities. In this work, we propose a simple and efficient method for creating realistic targeted synthetic datasets in the remote sensing domain, leveraging the opportunities offered by game development engines. We provide a description of the pipeline for procedural geometry generation and …
Sun Yat-Sen University（中山大学）；The Guangdong University of Technology（广东工业大学）
Land remote sensing analysis is a crucial research in earth science. In this work, we focus on a challenging task of land analysis, i.e., automatic extraction of traffic roads from remote sensing data, which has widespread applications in urban development and expansion estimation. Nevertheless, conventional methods either only utilized the limited information of aerial images, or simply fused multimodal information (e.g., vehicle trajectories), thus cannot well recognize unconstrained roads. To facilitate this problem, we introduce a novel neural network framework termed Cross-Modal Message P…
Image resolution is an important criterion for many applications based on satellite imagery. In this work, we adapt a state-of-the-art kernel regression technique for smartphone camera burst super-resolution to satellites. This technique leverages the local structure of the image to optimally steer the fusion kernels, limiting blur in the final high-resolution prediction, denoising the image, and recovering details up to a zoom factor of 2. We extend this approach to the multi-exposure case to predict from a sequence of multi-exposure low-resolution frames a high-resolution and noise-free one.…
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology & Hydrology（英国生态与水文中心）；University of Twente（特文特大学）
Assigning geospatial objects with specific categories at the pixel level is a fundamental task in remote sensing image analysis. Along with rapid development in sensor technologies, remotely sensed images can be captured at multiple spatial resolutions (MSR) with information content manifested at different scales. Extracting information from these MSR images represents huge opportunities for enhanced feature representation and characterisation. However, MSR images suffer from two critical issues: 1) increased scale variation of geo-objects and 2) loss of detailed information at coarse spatial …
Hochschule Ansbach（安斯巴赫应用技术大学）；Hochschule Darmstadt（达姆施塔特应用技术大学）；EURECOM（欧洲通信学院）；Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所）
Since early 2020 the COVID-19 pandemic has had a considerable impact on many aspects of daily life. A range of different measures have been implemented worldwide to reduce the rate of new infections and to manage the pressure on national health services. A primary strategy has been to reduce gatherings and the potential for transmission through the prioritisation of remote working and education. Enhanced hand hygiene and the use of facial masks have decreased the spread of pathogens when gatherings are unavoidable. These particular measures present challenges for reliable biometric recognition…
