OpenStreetMap (OSM) is a community-based, freely available, editable map service that was created as an alternative to authoritative ones. Given that it is edited mainly by volunteers with different mapping skills, the completeness and quality of its annotations are heterogeneous across different geographical locations. Despite that, OSM has been widely used in several applications in {Geosciences}, Earth Observation and environmental sciences. In this work, we present a review of recent methods based on machine learning to improve and use OSM data. Such methods aim either 1) at improving the coverage and quality of OSM layers, typically using GIS and remote sensing technologies, or 2) at using the existing OSM layers to train models based on image data to serve applications like navigation or {land use} classification. We believe that OSM (as well as other sources of open land maps) can change the way we interpret remote sensing data and that the synergy with machine learning can scale participatory map making and its quality to the level needed to serve global and up-to-date land mapping.
GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data
School of Earth and Space Sciences, Peking University（地球与空间科学学院，北京大学）；College of Urban and Environmental Sciences, Peking University（城市与环境科学学院，北京大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems Institute of Automation, CAS（多模态人工智能系统国家重点实验室，中国科学院自动化研究所）；Intelligent Maintenance and Operations Systems Lab（智能维护与运营系统实验室）
ONERA, The French Aerospace Lab（法国航空航天研究院）；Univ. Bretagne-Sud, UMR 6074, IRISA（南布列塔尼大学，IRISA）
In this work, we investigate the use of OpenStreetMap data for semantic labeling of Earth Observation images. Deep neural networks have been used in the past for remote sensing data classification from various sensors, including multispectral, hyperspectral, SAR and LiDAR data. While OpenStreetMap has already been used as ground truth data for training such networks, this abundant data source remains rarely exploited as an input information layer. In this paper, we study different use cases and deep network architectures to leverage OpenStreetMap data for semantic labeling of aerial and satell…
Research & Development Institute of Northwestern Polytechnical University in Shenzhen（西北工业大学深圳研究院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Computer Science, Wuhan University（武汉大学计算机学院）
Remote sensing image scene classification, which aims at labeling remote sensing images with a set of semantic categories based on their contents, has broad applications in a range of fields. Propelled by the powerful feature learning capabilities of deep neural networks, remote sensing image scene classification driven by deep learning has drawn remarkable attention and achieved significant breakthroughs. However, to the best of our knowledge, a comprehensive review of recent achievements regarding deep learning for scene classification of remote sensing images is still lacking. Considering t…
This paper provides an overview of how recent advances in machine learning and the availability of data from earth observing satellites can dramatically improve our ability to automatically map croplands over long period and over large regions. It discusses three applications in the domain of crop monitoring where ML approaches are beginning to show great promise. For each application, it highlights machine learning challenges, proposed approaches, and recent results. The paper concludes with discussion of major challenges that need to be addressed before ML approaches will reach their full po…
This paper introduces the task of visual question answering for remote sensing data (RSVQA). Remote sensing images contain a wealth of information which can be useful for a wide range of tasks including land cover classification, object counting or detection. However, most of the available methodologies are task-specific, thus inhibiting generic and easy access to the information contained in remote sensing data. As a consequence, accurate remote sensing product generation still requires expert knowledge. With RSVQA, we propose a system to extract information from remote sensing data that is a…
Technical University of Munich (TUM)（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心）
Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the past decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. But for many applications, such as urban population density mapping or urban utility planning, a classification map based on individual buildings is much more informative. However, such semantic classification still poses some fundamental challenges, for example, how to retrieve fine boundaries of individual buildings. In this paper, we proposed a general framework for classifying…
A Decade of Deep Learning for Remote Sensing Spatiotemporal Fusion: Advances, Challenges, and Opportunities
School of Computer Science, China University of Geosciences (Wuhan)（中国地质大学（武汉）计算机科学学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空航天信息研究所）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）
Scene classification is a fundamental task in interpretation of remote sensing images, and has become an active research topic in remote sensing community due to its important role in a wide range of applications. Over the past years, tremendous efforts have been made for developing powerful approaches for scene classification of remote sensing images, evolving from the traditional bag-of-visual-words model to the new generation deep convolutional neural networks (CNNs). The deep CNN based methods have exhibited remarkable breakthrough on performance, dramatically outperforming previous method…
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology & Hydrology（英国生态与水文中心）；University of Twente（特文特大学）
Assigning geospatial objects with specific categories at the pixel level is a fundamental task in remote sensing image analysis. Along with rapid development in sensor technologies, remotely sensed images can be captured at multiple spatial resolutions (MSR) with information content manifested at different scales. Extracting information from these MSR images represents huge opportunities for enhanced feature representation and characterisation. However, MSR images suffer from two critical issues: 1) increased scale variation of geo-objects and 2) loss of detailed information at coarse spatial …
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））
Fully automatic large-scale land cover mapping belongs to the core challenges addressed by the remote sensing community. Usually, the basis of this task is formed by (supervised) machine learning models. However, in spite of recent growth in the availability of satellite observations, accurate training data remains comparably scarce. On the other hand, numerous global land cover products exist and can be accessed often free-of-charge. Unfortunately, these maps are typically of a much lower resolution than modern day satellite imagery. Besides, they always come with a significant amount of nois…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
Data-driven methods such as convolutional neural networks (CNNs) are known to deliver state-of-the-art performance on image recognition tasks when the training data are abundant. However, in some instances, such as change detection in remote sensing images, annotated data cannot be obtained in sufficient quantities. In this work, we propose a simple and efficient method for creating realistic targeted synthetic datasets in the remote sensing domain, leveraging the opportunities offered by game development engines. We provide a description of the pipeline for procedural geometry generation and …
CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation
Technical University of Munich (TUM)（慕尼黑工业大学）；Remote Sensing Technology Institute, German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
Remote Sensing Based Crop Health Classification Using NDVI and Fully Connected Neural Networks
Sethu Institute of Technology（塞图理工学院）；College of Engineering and IT, University of Dubai（迪拜大学工程与信息技术学院）；Mohammed Bin Rashid Space Centre (MBRSC)（穆罕默德·本·拉希德航天中心）
A comprehensive review of remote sensing in wetland classification and mapping
Department of Geography, The University of Hong Kong（地理系，香港大学）；Department of Earth System Science, Tsinghua University（地球系统科学系，清华大学）；Department of Electronics and Information Engineering, Harbin Institute of Technology (Shenzhen)（电子与信息工程系，哈尔滨工业大学（深圳））；Pengcheng Laboratory（鹏城实验室）
Shen Yuan Honors College, Beihang University（北航沈元荣誉学院）；Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University（北京航空航天大学宇航学院航天智能科学与技术系）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北京航空航天大学虚拟现实技术与系统国家重点实验室）；Qian Xuesen Laboratory of Space Technology, China Academy of Space Technology（中国空间技术研究院钱学森空间技术实验室）
Mississippi State University（密西西比州立大学）；University of Malaya（马来亚大学）
In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of st…
Hochschule Ansbach（安斯巴赫应用技术大学）；Hochschule Darmstadt（达姆施塔特应用技术大学）；EURECOM（欧洲通信学院）；Fraunhofer Institute for Computer Graphics Research IGD（弗劳恩霍夫计算机图形学研究所）
Since early 2020 the COVID-19 pandemic has had a considerable impact on many aspects of daily life. A range of different measures have been implemented worldwide to reduce the rate of new infections and to manage the pressure on national health services. A primary strategy has been to reduce gatherings and the potential for transmission through the prioritisation of remote working and education. Enhanced hand hygiene and the use of facial masks have decreased the spread of pathogens when gatherings are unavoidable. These particular measures present challenges for reliable biometric recognition…
Vision-Language Models in Remote Sensing: Current Progress and Future Trends
King Abdullah University of Science and Technology（阿卜杜拉国王科技大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）；Peking University（北京大学）；Technical University of Munich（慕尼黑工业大学）；Munich Center for Machine Learning（慕尼黑机器学习中心）
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；International Research Center of Big Data for Sustainable Development Goals（可持续发展大数据国际研究中心）
Owing to the rapid development of sensor technology, hyperspectral (HS) remote sensing (RS) imaging has provided a significant amount of spatial and spectral information for the observation and analysis of the Earth's surface at a distance of data acquisition devices, such as aircraft, spacecraft, and satellite. The recent advancement and even revolution of the HS RS technique offer opportunities to realize the full potential of various applications, while confronting new challenges for efficiently processing and analyzing the enormous HS acquisition data. Due to the maintenance of the 3-D HS …
FlexiMo: A Flexible Remote Sensing Foundation Model
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；School of Mathematics and Statistics, Southeast University（东南大学数学学院）；Helmholtz-Zentrum Dresden-Rossendorf (HZDR)（德累斯顿-罗森多夫亥姆霍兹中心）；Lancaster Environment Centre, Lancaster University（兰卡斯特大学兰卡斯特环境中心）
AI Security for Geoscience and Remote Sensing: Challenges and Future Trends
Institute of Advanced Research in Artificial Intelligence（人工智能高等研究院）；Nanyang Technological University（南洋理工大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Lancaster University（兰卡斯特大学）；University of Southampton（南安普顿大学）
LithoBench: Benchmarking Large Multimodal Models for Remote-Sensing Lithology Interpretation
School of Computer Science, China University of Geosciences（中国地质大学（北京）计算机学院）；PRADA Lab, King Abdullah University of Science and Technology（科廷大学PRADA实验室）；Department of Computer Science, University of Exeter（埃克塞特大学计算机科学系）
Wageningen University（瓦赫宁根大学）；Ecole Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；University of Bonn（波恩大学）；ETH Zurich（苏黎世联邦理工学院）；University of Kentucky（肯塔基大学）；Technical University of Munich（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）；Universitat de València（瓦伦西亚大学）
In the last years we have witnessed the fields of geosciences and remote sensing and artificial intelligence to become closer. Thanks to both the massive availability of observational data, improved simulations, and algorithmic advances, these disciplines have found common objectives and challenges to advance the modeling and understanding of the Earth system. Despite such great opportunities, we also observed a worrying tendency to remain in disciplinary comfort zones applying recent advances from artificial intelligence on well resolved remote sensing problems. Here we take a position on res…
Federated Learning Across Decentralized and Unshared Archives for Remote Sensing Image Classification
Technische Universität Berlin（柏林工业大学）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）；École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）
Deep learning (DL) has seen great success in the computer vision (CV) field, and related techniques have been used in security, healthcare, remote sensing, and many other fields. As a parallel development, visual data has become universal in daily life, easily generated by ubiquitous low-cost cameras. Therefore, exploring DL-based CV may yield useful information about objects, such as their number, locations, distribution, motion, etc. Intuitively, DL-based CV can also facilitate and improve the designs of wireless communications, especially in dynamic network scenarios. However, so far, such …
Applications of Knowledge Distillation in Remote Sensing: A Survey
College of Engineering and Information Technology, University of Dubai（迪拜大学工程与信息技术学院）；University of Dubai（迪拜大学）；United Arab Emirates University（阿联酋大学）；Harokopio University of Athens（雅典哈科皮奥大学）
Deep Learning for Wildfire Risk Prediction: Integrating Remote Sensing and Environmental Data
Schulich School of Engineering, Department of Geomatics Engineering, University of Calgary（Schulich 工程学院，测绘工程系，卡尔加里大学）；Department of Geography and Environmental Management, University of Waterloo（地理与环境管理系，滑铁卢大学）；Department of Systems Design Engineering, University of Waterloo（系统设计工程系，滑铁卢大学）；School of Emergency Management, Nanjing University of Information Science and Technology（应急管理学院，南京信息工程大学）；Division of Geoinformatics, KTH Royal Institute of Technology（地理信息学系，皇家理工学院）；School of Remote Sensing and Geomatics Engineering, Nanjing University of Information Science and Technology（遥感与测绘工程学院，南京信息工程大学）；Department of Software Engineering, University of Waterloo（软件工程系，滑铁卢大学）；Department of Management Science and Engineering, University of Waterloo（管理科学与工程系，滑铁卢大学）
Massive amounts of satellite data have been gathered over time, holding the potential to unveil a spatiotemporal chronicle of the surface of Earth. These data allow scientists to investigate various important issues, such as land use changes, on a global scale. However, not all land-use phenomena are equally visible on satellite imagery. In particular, the creation of an inventory of the planet's road infrastructure remains a challenge, despite being crucial to analyze urbanization patterns and their impact. Towards this end, this work advances data-driven approaches for the automatic identifi…
Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality
School of Automation, Southeast University（自动化学院，东南大学）；School of Mathematics, Southeast University（数学学院，东南大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（航天信息研究所，中国科学院）；Image Processing Laboratory (IPL), Universitat de València（图像处理实验室（IPL），瓦伦西亚大学）；Univ. Grenoble Alpes, INRIA, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学，INRIA，CNRS，格勒诺布尔INP，LJK）
Columbia University（哥伦比亚大学）；University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
To extract information at scale, researchers increasingly apply semantic segmentation techniques to remotely-sensed imagery. While fully-supervised learning enables accurate pixel-wise segmentation, compiling the exhaustive datasets required is often prohibitively expensive. As a result, many non-urban settings lack the ground-truth needed for accurate segmentation. Existing open source infrastructure data for these regions can be inexact and non-exhaustive. Open source infrastructure annotations like OpenStreetMaps (OSM) are representative of this issue: while OSM labels provide global insigh…
Stanford University（斯坦福大学）；IBM Research Europe（IBM欧洲研究院）；IBM Europe（IBM欧洲）
Current methods to determine the energy efficiency of buildings require on-site visits of certified energy auditors which makes the process slow, costly, and geographically incomplete. To accelerate the identification of promising retrofit targets on a large scale, we propose to estimate building energy efficiency from widely available and remotely sensed data sources only, namely street view, aerial view, footprint, and satellite-borne land surface temperature (LST) data. After collecting data for almost 40,000 buildings in the United Kingdom, we combine these data sources by training multipl…
