This work explores the combination of free cloud computing, free open-source software, and deep learning methods to analyse a real, large-scale problem: the automatic country-wide identification and classification of surface mines and mining tailings dams in Brazil. Locations of officially registered mines and dams were obtained from the Brazilian government open data resource. Multispectral Sentinel-2 satellite imagery, obtained and processed at the Google Earth Engine platform, was used to train and test deep neural networks using the TensorFlow 2 API and Google Colab platform. Fully Convolutional Neural Networks were used in an innovative way, to search for unregistered ore mines and tailing dams in large areas of the Brazilian territory. The efficacy of the approach is demonstrated by the discovery of 263 mines that do not have an official mining concession. This exploratory work highlights the potential of a set of new technologies, freely available, for the construction of low cost data science tools that have high social impact. At the same time, it discusses and seeks to suggest practical solutions for the complex and serious problem of illegal mining and the proliferation of tailings dams, which pose high risks to the population and the environment, especially in developing countries. Code is made publicly available at: https://github.com/remis/mining-discovery-with-deep-learning.
Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）；Universidade Católica de Brasília（巴西利亚天主教大学）；Tribunal de Contas da União（联邦审计法院）
In this work we present BrazilDAM, a novel public dataset based on Sentinel-2 and Landsat-8 satellite images covering all tailings dams cataloged by the Brazilian National Mining Agency (ANM). The dataset was built using georeferenced images from 769 dams, recorded between 2016 and 2019. The time series were processed in order to produce cloud free images. The dams contain mining waste from different ore categories and have highly varying shapes, areas and volumes, making BrazilDAM particularly interesting and challenging to be used in machine learning benchmarks. The original catalog contains…
Monitoring dam failures using satellite images provides first responders with efficient management of early interventions. It is also equally important to monitor spatial and temporal changes in the inundation area to track the post-disaster recovery. On January 25th, 2019, the tailings dam of the Córrego do Feijão iron ore mine, located in Brumadinho, Brazil, collapsed. This disaster caused more than 230 fatalities and 30 missing people leading to damage on the order of multiple billions of dollars. This study uses Sentinel-2 satellite images to map the inundation area and assess and delineat…
Google AI（谷歌人工智能）；UN World Food Programme（联合国世界粮食计划署）
In all types of disasters, from earthquakes to armed conflicts, aid workers need accurate and timely data such as damage to buildings and population displacement to mount an effective response. Remote sensing provides this data at an unprecedented scale, but extracting operationalizable information from satellite images is slow and labor-intensive. In this work, we use machine learning to automate the detection of building damage in satellite imagery. We compare the performance of four different convolutional neural network models in detecting damaged buildings in the 2010 Haiti earthquake. We…
Skolkovo institute of Science and technology（斯科尔科沃科学技术研究院）；Moscow（莫斯科）
When major disaster occurs the questions are raised how to estimate the damage in time to support the decision making process and relief efforts by local authorities or humanitarian teams. In this paper we consider the use of Machine Learning and Computer Vision on remote sensing imagery to improve time efficiency of assessment of damaged buildings in disaster affected area. We propose a general workflow that can be useful in various disaster management applications, and demonstrate the use of the proposed workflow for the assessment of the damage caused by the wildfires in California in 2017.
Gaining timely and reliable situation awareness after hazard events such as a hurricane is crucial to emergency managers and first responders. One effective way to achieve that goal is through damage assessment. Recently, disaster researchers have been utilizing imagery captured through satellites or drones to quantify the number of flooded/damaged buildings. In this paper, we propose a mixed data approach, which leverages publicly available satellite imagery and geolocation features of the affected area to identify damaged buildings after a hurricane. The method demonstrated significant impro…
Google Research（谷歌研究院）；Stanford University（斯坦福大学）；United Nations World Food Programme（联合国世界粮食计划署）
To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected. Recent research combines machine learning with remote sensing to automatically extract such information from satellite imagery, reducing manual labor and turn-around time. A major impediment to using machine learning methods in real disaster response scenarios is the difficulty of obtaining a sufficient amount of labeled data to train a model for a…
Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles…
Carnegie Mellon University（卡内基梅隆大学）；Software Engineering Institute（软件工程研究所）；Defense Innovation Unit（国防创新部门）；Department of Defense（国防部）；CrowdAI, Inc.（CrowdAI公司）
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exists for using aerial imagery combined with computer vision algorithms to assess damage and reduce the potential danger to human life. In collaboration with multiple disaster response agencies, xBD prov…
In this manuscript, an image analytics based deep learning framework for wind turbine blade surface damage detection is proposed. Turbine blade(s) which carry approximately one-third of a turbine weight are susceptible to damage and can cause sudden malfunction of a grid-connected wind energy conversion system. The surface damage detection of wind turbine blade requires a large dataset so as to detect a type of damage at an early stage. Turbine blade images are captured via aerial imagery. Upon inspection, it is found that the image dataset was limited and hence image augmentation is applied t…
After a hurricane, damage assessment is critical to emergency managers for efficient response and resource allocation. One way to gauge the damage extent is to quantify the number of flooded/damaged buildings, which is traditionally done by ground survey. This process can be labor-intensive and time-consuming. In this paper, we propose to improve the efficiency of building damage assessment by applying image classification algorithms to post-hurricane satellite imagery. At the known building coordinates (available from public data), we extract square-sized images from the satellite imagery to …
Post-hurricane damage assessment is crucial towards managing resource allocations and executing an effective response. Traditionally, this evaluation is performed through field reconnaissance, which is slow, hazardous, and arduous. Instead, in this paper we furthered the idea of implementing deep learning through convolutional neural networks in order to classify post-hurricane satellite imagery of buildings as Flooded/Damaged or Undamaged. The experimentation was conducted employing a dataset containing post-hurricane satellite imagery from the Greater Houston area after Hurricane Harvey in 2…
Massachusetts Institue of Technology（麻省理工学院）；WL Research
Automatic change detection and disaster damage assessment are currently procedures requiring a huge amount of labor and manual work by satellite imagery analysts. In the occurrences of natural disasters, timely change detection can save lives. In this work, we report findings on problem framing, data processing and training procedures which are specifically helpful for the task of building damage assessment using the newly released xBD dataset. Our insights lead to substantial improvement over the xBD baseline models, and we score among top results on the xView2 challenge leaderboard. We relea…
Purdue University（普渡大学）；Lockheed Martin Space（洛克希德·马丁太空公司）
When disaster strikes, accurate situational information and a fast, effective response are critical to save lives. Widely available, high resolution satellite images enable emergency responders to estimate locations, causes, and severity of damage. Quickly and accurately analyzing the extensive amount of satellite imagery available, though, requires an automatic approach. In this paper, we present Siam-U-Net-Attn model - a multi-class deep learning model with an attention mechanism - to assess damage levels of buildings given a pair of satellite images depicting a scene before and after a disa…
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Nanjing University of Science and Technology（南京理工大学）
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before an effective response is conducted. High-resolution satellite images provide rich information with pre- and post-disaster scenes for analysis. However, most existing works simply use pre- and post-disaster images as input without considering their correlations. In this paper, we propose a novel cross-directional fusion strategy to better explore the correlations between pre- and post-disaster images. Moreover, the da…
New York University（纽约大学）；University of the Philippines Diliman（菲律宾大学迪里曼分校）；University of Washington（华盛顿大学）；German Corporation for International Cooperation GmbH (GIZ)（德国国际合作机构（GIZ））
Rapid damage assessment is of crucial importance to emergency responders during hurricane events, however, the evaluation process is often slow, labor-intensive, costly, and error-prone. New advances in computer vision and remote sensing open possibilities to observe the Earth at a different scale. However, substantial pre-processing work is still required in order to apply state-of-the-art methodology for emergency response. To enable the comparison of methods for automatic detection of damaged buildings from post-hurricane remote sensing imagery taken from both airborne and satellite sensors…
King’s College London（伦敦国王学院）；UN Environment World Conservation Monitoring Centre（联合国环境规划署世界保护监测中心）
Dam reservoirs play an important role in meeting sustainable development goals and global climate targets. However, particularly for small dam reservoirs, there is a lack of consistent data on their geographical location. To address this data gap, a promising approach is to perform automated dam reservoir extraction based on globally available remote sensing imagery. It can be considered as a fine-grained task of water body extraction, which involves extracting water areas in images and then separating dam reservoirs from natural water bodies. We propose a novel deep neural network (DNN) based…
The Academy for Mathematics, Science, and Engineering（数学、科学与工程学院）
Natural disasters ravage the world's cities, valleys, and shores on a regular basis. Deploying precise and efficient computational mechanisms for assessing infrastructure damage is essential to channel resources and minimize the loss of life. Using a dataset that includes labeled pre- and post- disaster satellite imagery, we take a machine learning-based remote sensing approach and train multiple convolutional neural networks (CNNs) to assess building damage on a per-building basis. We present a novel methodology of interpretable deep learning that seeks to explicitly investigate the most usef…
Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning
Built Environment Department, College of Science and Technology, North Carolina A&T State University（北卡罗来纳农工州立大学科技学院建筑环境系）；United Nations University Institute for Water, Environment and Health（联合国大学水、环境与健康研究所）
Multiclass Post-Earthquake Building Assessment Integrating High-Resolution Optical and SAR Satellite Imagery, Ground Motion, and Soil Data with Transformers
Department of Civil and Environmental Engineering, University of Houston, TX, USA（土木与环境工程系，德克萨斯大学）；Department of Electrical and Computer Engineering, University of Houston, TX, USA（电气与计算机工程系，德克萨斯大学）；Department of Earth and Atmospheric Science, University of Houston, TX, USA（地球与大气科学系，德克萨斯大学）；German Aerospace Center (DLR), Microwaves and Radar Institute, Munich, Germany（德国航天中心（DLR）微波与雷达研究所）
Puerto Rico suffered severe damage from the category 5 hurricane (Maria) in September 2017. Total monetary damages are estimated to be ~92 billion USD, the third most costly tropical cyclone in US history. The response to this damage has been tempered and slow moving, with recent estimates placing 45% of the population without power three months after the storm. Consequently, we developed a unique data-fusion mapping approach called the Urban Development Index (UDI) and new open source tool, Comet Time Series (CometTS), to analyze the recovery of electricity and infrastructure in Puerto Rico. …
Bridge Damage Detection from Low-Light UAV Imagery via Degradation-Aware Mixture-of-Experts Enhancement
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Sustainability X-Lab, The University of Hong Kong（香港大学可持续发展X实验室）；School of Architecture, Building, and Civil Engineering, Loughborough University（拉夫堡大学建筑、建造与土木工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科技大学土木与环境工程系）
Lancaster University（兰卡斯特大学）；University of Bath（巴斯大学）；University of Oxford（牛津大学）
High-resolution satellite imagery available immediately after disaster events is crucial for response planning as it facilitates broad situational awareness of critical infrastructure status such as building damage, flooding, and obstructions to access routes. Damage mapping at this scale would require hundreds of expert person-hours. However, a combination of crowdsourcing and recent advances in deep learning reduces the effort needed to just a few hours in real time. Asking volunteers to place point marks, as opposed to shapes of actual damaged areas, significantly decreases the required ana…
Renmin University of China（中国人民大学）；Connected Robotics Inc.（互联机器人公司）；RIKEN Center for Advanced Intelligence Project (AIP)（理化学研究所先进智能项目中心）
In the field of post-disaster assessment, for timely and accurate rescue and localization after a disaster, people need to know the location of damaged buildings. In deep learning, some scholars have proposed methods to make automatic and highly accurate building damage assessments by remote sensing images, which are proved to be more efficient than assessment by domain experts. However, due to the lack of a large amount of labeled data, these kinds of tasks can suffer from being able to do an accurate assessment, as the efficiency of deep learning models relies highly on labeled data. Althoug…
ELTE Eötvös Loránd University（厄特沃什·罗兰大学（ELTE））；Tisza Plastic Cup Initiative（蒂萨塑料杯倡议组织）
One of the biggest environmental problems of our time is the increase in illegal landfills in forests, rivers, on river banks and other secluded places. In addition, waste in rivers causes damage not only locally, but also downstream, both in the water and washed ashore. Large islands of waste can also form at hydroelectric power stations and dams, and if they continue to flow, they can cause further damage to the natural environment along the river. Recent studies have also proved that rivers are the main source of plastic pollution in marine environments. Monitoring potential sources of dang…
COMSATS University Islamabad（伊斯兰堡COMSATS大学）；University of Tasmania（塔斯马尼亚大学）；National University of Ireland, Galway（爱尔兰国立高威大学）
Food resources face severe damages under extraordinary situations of catastrophes such as earthquakes, cyclones, and tsunamis. Under such scenarios, speedy assessment of food resources from agricultural land is critical as it supports aid activity in the disaster hit areas. In this article, a deep learning approach is presented for the detection and segmentation of coconut tress in aerial imagery provided through the AI competition organized by the World Bank in collaboration with OpenAerialMap and WeRobotics. Maked Region-based Convolutional Neural Network approach was used identification and…
Institute of Computer Science（计算机科学研究所）；Campus Institute Data Science（校园数据科学研究所）；University of Goettingen（哥廷根大学）
An important step for limiting the negative impact of natural disasters is rapid damage assessment after a disaster occurred. For instance, building damage detection can be automated by applying computer vision techniques to satellite imagery. Such models operate in a multi-domain setting: every disaster is inherently different (new geolocation, unique circumstances), and models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. Accordingly, estimating real-world performance requires an out-of-domain (OOD) test set. Howeve…
Video and Image Processing Laboratory（视频与图像处理实验室）；School of Electrical Engineering（电气工程学院）；Purdue University（普渡大学）
Satellite imagery is becoming increasingly accessible due to the growing number of orbiting commercial satellites. Many applications make use of such images: agricultural management, meteorological prediction, damage assessment from natural disasters, or cartography are some of the examples. Unfortunately, these images can be easily tampered and modified with image manipulation tools damaging downstream applications. Because the nature of the manipulation applied to the image is typically unknown, unsupervised methods that don't require prior knowledge of the tampering techniques used are pref…
Università di Corsica（科西嘉大学）；Tecnológico de Monterrey（蒙特雷理工学院）
Early wildfire detection is of paramount importance to avoid as much damage as possible to the environment, properties, and lives. Deep Learning (DL) models that can leverage both visible and infrared information have the potential to display state-of-the-art performance, with lower false-positive rates than existing techniques. However, most DL-based image fusion methods have not been evaluated in the domain of fire imagery. Additionally, to the best of our knowledge, no publicly available dataset contains visible-infrared fused fire images. There is a growing interest in DL-based image fusio…
Deep learning has led to many recent advances in object detection and instance segmentation, among other computer vision tasks. These advancements have led to wide application of deep learning based methods and related methodologies in object detection tasks for satellite imagery. In this paper, we introduce MIS Check-Dam, a new dataset of check-dams from satellite imagery for building an automated system for the detection and mapping of check-dams, focusing on the importance of irrigation structures used for agriculture. We review some of the most recent object detection and instance segmenta…
North Carolina State University（北卡罗来纳州立大学）；Colaberry Inc（Colaberry公司）；Iowa State University（爱荷华州立大学）
Severe weather events can cause large financial losses to farmers. Detailed information on the location and severity of damage will assist farmers, insurance companies, and disaster response agencies in making wise post-damage decisions. The goal of this study was a proof-of-concept to detect damaged crop areas from aerial imagery using computer vision and deep learning techniques. A specific objective was to compare existing object detection algorithms to determine which was best suited for crop damage detection. Two modes of crop damage common in maize (corn) production were simulated: stalk…
Lebanese University（黎巴嫩大学）；Islamic University of Lebanon（黎巴嫩伊斯兰大学）；MINES ParisTech（巴黎高科矿业学院）；National Center for Remote Sensing - CNRS（法国国家科学研究中心国家遥感中心）
Buildings classification using satellite images is becoming more important for several applications such as damage assessment, resource allocation, and population estimation. We focus, in this work, on buildings damage assessment (BDA) and buildings type classification (BTC) of residential and non-residential buildings. We propose to rely solely on RGB satellite images and follow a 2-stage deep learning-based approach, where first, buildings' footprints are extracted using a semantic segmentation model, followed by classification of the cropped images. Due to the lack of an appropriate dataset…
Timely disaster risk management requires accurate road maps and prompt damage assessment. Currently, this is done by volunteers manually marking satellite imagery of affected areas but this process is slow and often error-prone. Segmentation algorithms can be applied to satellite images to detect road networks. However, existing methods are unsuitable for disaster-struck areas as they make assumptions about the road network topology which may no longer be valid in these scenarios. Herein, we propose a CNN-based framework for identifying accessible roads in post-disaster imagery by detecting ch…
Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN
Technical University of Munich（慕尼黑工业大学）；University of the Bundeswehr Munich（慕尼黑联邦国防军大学）；China University of Mining and Technology（中国矿业大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）；Tongji University（同济大学）
A Deep Learning framework for building damage assessment using VHR SAR and geospatial data: demonstration on the 2023 Turkiye Earthquake
Department of Electrical, Computer and Biomedical Engineering, University of Pavia（帕维亚大学电气、计算机与生物医学工程系）；Department of Engineering, University of Sannio（萨诺尼奥大学工程系）；Italian Space Agency (ASI)（意大利航天局）
Improved MambdaBDA Framework for Robust Building Damage Assessment Across Disaster Domains
Department of Computer Engineering, Istanbul Technical University（计算机工程系，伊斯坦布尔技术大学）；Division of Engineering, New York University Abu Dhabi（工程学院，纽约大学阿布扎比分校）
Within commercial wind energy generation, the monitoring and predictive maintenance of wind turbine blades in-situ is a crucial task, for which remote monitoring via aerial survey from an Unmanned Aerial Vehicle (UAV) is commonplace. Turbine blades are susceptible to both operational and weather-based damage over time, reducing the energy efficiency output of turbines. In this study, we address automating the otherwise time-consuming task of both blade detection and extraction, together with fault detection within UAV-captured turbine blade inspection imagery. We propose BladeNet, an applicati…
Wildfires have increased in frequency and severity over the past two decades, especially in the Western United States. Beyond physical infrastructure damage caused by these wildfire events, researchers have increasingly identified harmful impacts of particulate matter generated by wildfire smoke on respiratory, cardiovascular, and cognitive health. This inference is difficult due to the spatial and temporal uncertainty regarding how much particulate matter is specifically attributable to wildfire smoke. One factor contributing to this challenge is the reliance on manually drawn smoke plume ann…
The use of satellite imagery has become increasingly popular for disaster monitoring and response. After a disaster, it is important to prioritize rescue operations, disaster response and coordinate relief efforts. These have to be carried out in a fast and efficient manner since resources are often limited in disaster-affected areas and it's extremely important to identify the areas of maximum damage. However, most of the existing disaster mapping efforts are manual which is time-consuming and often leads to erroneous results. In order to address these issues, we propose a framework for chang…
The Amazon rainforests have been suffering widespread damage, both via natural and artificial means. Every minute, it is estimated that the world loses forest cover the size of 48 football fields. Deforestation in the Amazon rainforest has led to drastically reduced biodiversity, loss of habitat, climate change, and other biological losses. In this respect, it has become essential to track how the nature of these forests change over time. Image classification using deep learning can help speed up this process by removing the manual task of classifying each image. Here, it is shown how convolut…
Thermal and RGB Images Work Better Together in Wind Turbine Damage Detection
Khmelnytskyi National University（赫梅利尼茨基国立大学）；West Ukrainian National University（西乌克兰国立大学）；Kazimierz Pulaski University of Technology and Humanities in Radom（拉多姆卡齐米日·普瓦斯基科技与人文大学）
University of Oxford（牛津大学）；TU Munich（慕尼黑工业大学）；University of Kent（肯特大学）；LIST Luxembourg（卢森堡LIST研究所）；DFKI（德国人工智能研究中心）；TU Kaiserslautern（凯泽斯劳滕工业大学）；Czech Geological Survey（捷克地质调查局）；University of Warsaw（华沙大学）
We propose a novel approach for rapid segmentation of flooded buildings by fusing multiresolution, multisensor, and multitemporal satellite imagery in a convolutional neural network. Our model significantly expedites the generation of satellite imagery-based flood maps, crucial for first responders and local authorities in the early stages of flood events. By incorporating multitemporal satellite imagery, our model allows for rapid and accurate post-disaster damage assessment and can be used by governments to better coordinate medium- and long-term financial assistance programs for affected ar…
Northern Arizona University（北亚利桑那大学）；Air Force Research Laboratory（美国空军研究实验室）
Wildfires are one of the costliest and deadliest natural disasters in the US, causing damage to millions of hectares of forest resources and threatening the lives of people and animals. Of particular importance are risks to firefighters and operational forces, which highlights the need for leveraging technology to minimize danger to people and property. FLAME (Fire Luminosity Airborne-based Machine learning Evaluation) offers a dataset of aerial images of fires along with methods for fire detection and segmentation which can help firefighters and researchers to develop optimal fire management …
