To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected. Recent research combines machine learning with remote sensing to automatically extract such information from satellite imagery, reducing manual labor and turn-around time. A major impediment to using machine learning methods in real disaster response scenarios is the difficulty of obtaining a sufficient amount of labeled data to train a model for an unfolding disaster. This paper shows a novel application of semi-supervised learning (SSL) to train models for damage assessment with a minimal amount of labeled data and large amount of unlabeled data. We compare the performance of state-of-the-art SSL methods, including MixMatch and FixMatch, to a supervised baseline for the 2010 Haiti earthquake, 2017 Santa Rosa wildfire, and 2016 armed conflict in Syria. We show how models trained with SSL methods can reach fully supervised performance despite using only a fraction of labeled data and identify areas for further improvements.
Lancaster University（兰卡斯特大学）；University of Bath（巴斯大学）；University of Oxford（牛津大学）
High-resolution satellite imagery available immediately after disaster events is crucial for response planning as it facilitates broad situational awareness of critical infrastructure status such as building damage, flooding, and obstructions to access routes. Damage mapping at this scale would require hundreds of expert person-hours. However, a combination of crowdsourcing and recent advances in deep learning reduces the effort needed to just a few hours in real time. Asking volunteers to place point marks, as opposed to shapes of actual damaged areas, significantly decreases the required ana…
In the aftermath of disasters, building damage maps are obtained using change detection to plan rescue operations. Current convolutional neural network approaches do not consider the similarities between neighboring buildings for predicting the damage. We present a novel graph-based building damage detection solution to capture these relationships. Our proposed model architecture learns from both local and neighborhood features to predict building damage. Specifically, we adopt the sample and aggregate graph convolution strategy to learn aggregation functions that generalize to unseen graphs w…
Research Institute for Infrastructure Paradigm Shift, Yachiyo Engineering Co., Ltd.（八千代工程公司基础设施范式转变研究所）
Recent years, typhoon damages has become social problem owing to climate change. In 9 September 2019, Typhoon Faxai passed on the Chiba in Japan, whose damages included with electric provision stop because of strong wind recorded on the maximum 45 meter per second. A large amount of tree fell down, and the neighbor electric poles also fell down at the same time. These disaster features have caused that it took 18 days for recovery longer than past ones. Immediate responses are important for faster recovery. As long as we can, aerial survey for global screening of devastated region would be req…
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before relief effort is deployed. With a pair of pre- and post-disaster satellite images, building damage assessment aims at predicting the extent of damage to buildings. With the powerful ability of feature representation, deep neural networks have been successfully applied to building damage assessment. Most existing works simply concatenate pre- and post-disaster images as input of a deep neural network without consideri…
The use of satellite imagery has become increasingly popular for disaster monitoring and response. After a disaster, it is important to prioritize rescue operations, disaster response and coordinate relief efforts. These have to be carried out in a fast and efficient manner since resources are often limited in disaster-affected areas and it's extremely important to identify the areas of maximum damage. However, most of the existing disaster mapping efforts are manual which is time-consuming and often leads to erroneous results. In order to address these issues, we propose a framework for chang…
RIKEN Center for Advance Intelligence Project（理化学研究所革新智能研究中心）；the University of Tokyo（东京大学）；Hiroshima University（广岛大学）；Chiba University（千叶大学）；Tokyo Institute of Technology（东京工业大学）；Tohoku University（东北大学）
Earth observation technologies, such as optical imaging and synthetic aperture radar (SAR), provide excellent means to monitor ever-growing urban environments continuously. Notably, in the case of large-scale disasters (e.g., tsunamis and earthquakes), in which a response is highly time-critical, images from both data modalities can complement each other to accurately convey the full damage condition in the disaster's aftermath. However, due to several factors, such as weather and satellite coverage, it is often uncertain which data modality will be the first available for rapid disaster respo…
Automatic post-disaster damage detection using aerial imagery is crucial for quick assessment of damage caused by disaster and development of a recovery plan. The main problem preventing us from creating an applicable model in practice is that damaged (positive) examples we are trying to detect are much harder to obtain than undamaged (negative) examples, especially in short time. In this paper, we revisit the classical bootstrap aggregating approach in the context of modern transfer learning for data-efficient disaster damage detection. Unlike previous classical ensemble learning articles, ou…
Massachusetts Institute of Technology（麻省理工学院）；MIT Lincoln Laboratory（麻省理工学院林肯实验室）
Aerial images provide important situational awareness for responding to natural disasters such as hurricanes. They are well-suited for providing information for damage estimation and localization (DEL); i.e., characterizing the type and spatial extent of damage following a disaster. Despite recent advances in sensing and unmanned aerial systems technology, much of post-disaster aerial imagery is still taken by handheld DSLR cameras from small, manned, fixed-wing aircraft. However, these handheld cameras lack IMU information, and images are taken opportunistically post-event by operators. As su…
Massachusetts Institue of Technology（麻省理工学院）；WL Research
Automatic change detection and disaster damage assessment are currently procedures requiring a huge amount of labor and manual work by satellite imagery analysts. In the occurrences of natural disasters, timely change detection can save lives. In this work, we report findings on problem framing, data processing and training procedures which are specifically helpful for the task of building damage assessment using the newly released xBD dataset. Our insights lead to substantial improvement over the xBD baseline models, and we score among top results on the xView2 challenge leaderboard. We relea…
Decoding the Disaster: Multi-Task Geospatial Reasoning with Vision-Language Models and Crowdsourced Imagery for Disaster Mapping
Shandong University of Science and Technology（山东科技大学）；Technical University of Munich（慕尼黑工业大学）；Wuhan University（武汉大学）；Tsinghua University（清华大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；University of Iceland（冰岛大学）
Kansas State University（堪萨斯州立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Traditional post-disaster assessment of damage heavily relies on expensive GIS data, especially remote sensing image data. In recent years, social media has become a rich source of disaster information that may be useful in assessing damage at a lower cost. Such information includes text (e.g., tweets) or images posted by eyewitnesses of a disaster. Most of the existing research explores the use of text in identifying situational awareness information useful for disaster response teams. The use of social media images to assess disaster damage is limited. In this paper, we propose a novel appro…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Visual Question Answering system integrated with Unmanned Aerial Vehicle (UAV) has a lot of potentials to advance the post-disaster damage assessment purpose. Providing assistance to affected areas is highly dependent on real-time data assessment and analysis. Scope of the Visual Question Answering is to understand the scene and provide query related answer which certainly faster the recovery process after any disaster. In this work, we address the importance of \textit{visual question answering (VQA)} task for post-disaster damage assessment by presenting our recently developed VQA dataset ca…
Google AI（谷歌人工智能）；UN World Food Programme（联合国世界粮食计划署）
In all types of disasters, from earthquakes to armed conflicts, aid workers need accurate and timely data such as damage to buildings and population displacement to mount an effective response. Remote sensing provides this data at an unprecedented scale, but extracting operationalizable information from satellite images is slow and labor-intensive. In this work, we use machine learning to automate the detection of building damage in satellite imagery. We compare the performance of four different convolutional neural network models in detecting damaged buildings in the 2010 Haiti earthquake. We…
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（德克萨斯A&M大学）
In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）
The detrimental impacts of climate change include stronger and more destructive hurricanes happening all over the world. Identifying different damaged structures of an area including buildings and roads are vital since it helps the rescue team to plan their efforts to minimize the damage caused by a natural disaster. Semantic segmentation helps to identify different parts of an image. We implement a novel self-attention based semantic segmentation model on a high resolution UAV dataset and attain Mean IoU score of around 88% on the test set. The result inspires to use self-attention schemes in…
Skolkovo institute of Science and technology（斯科尔科沃科学技术研究院）；Moscow（莫斯科）
When major disaster occurs the questions are raised how to estimate the damage in time to support the decision making process and relief efforts by local authorities or humanitarian teams. In this paper we consider the use of Machine Learning and Computer Vision on remote sensing imagery to improve time efficiency of assessment of damaged buildings in disaster affected area. We propose a general workflow that can be useful in various disaster management applications, and demonstrate the use of the proposed workflow for the assessment of the damage caused by the wildfires in California in 2017.
Hyperlocal disaster damage assessment using bi-temporal street-view imagery and pre-trained vision models
Department of Geography, Texas A&M University（地理系，德克萨斯A&M大学）；Department of Geography, Pennsylvania State University（地理系，宾夕法尼亚州立大学）；Department of Geography and Sustainability, University of Tennessee（地理与可持续发展系，田纳西大学）；Spatial Sciences Institute, University of Southern California（空间科学研究所，南加州大学）
This study aims to enable more reliable automated post-disaster building damage classification using artificial intelligence (AI) and multi-view imagery. The current practices and research efforts in adopting AI for post-disaster damage assessment are generally (a) qualitative, lacking refined classification of building damage levels based on standard damage scales, and (b) trained based on aerial or satellite imagery with limited views, which, although indicative, are not completely descriptive of the damage scale. To enable more accurate and reliable automated quantification of damage levels…
Institute of Computer Science（计算机科学研究所）；Campus Institute Data Science（校园数据科学研究所）；University of Goettingen（哥廷根大学）
An important step for limiting the negative impact of natural disasters is rapid damage assessment after a disaster occurred. For instance, building damage detection can be automated by applying computer vision techniques to satellite imagery. Such models operate in a multi-domain setting: every disaster is inherently different (new geolocation, unique circumstances), and models must be robust to a shift in distribution between disaster imagery available for training and the images of the new event. Accordingly, estimating real-world performance requires an out-of-domain (OOD) test set. Howeve…
Humanitarian disasters and political violence cause significant damage to our living space. The reparation cost to homes, infrastructure, and the ecosystem is often difficult to quantify in real-time. Real-time quantification is critical to both informing relief operations, but also planning ahead for rebuilding. Here, we use satellite images before and after major crisis around the world to train a robust baseline Residual Network (ResNet) and a disaster quantification Pyramid Scene Parsing Network (PSPNet). ResNet offers robustness to poor image quality and can identify areas of destruction …
Institute for Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））；Semnan University（塞姆南大学）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Social media plays a significant role in sharing essential information, which helps humanitarian organizations in rescue operations during and after disaster incidents. However, developing an efficient method that can provide rapid analysis of social media images in the early hours of disasters is still largely an open problem, mainly due to the lack of suitable datasets and the sheer complexity of this task. In addition, supervised methods can not generalize well to novel disaster incidents. In this paper, inspired by the success of Knowledge Distillation (KD) methods, we propose an unsupervi…
In this paper, we study the problem of efficiently assessing building damage after natural disasters like hurricanes, floods or fires, through aerial video analysis. We make two main contributions. The first contribution is a new dataset, consisting of user-generated aerial videos from social media with annotations of instance-level building damage masks. This provides the first benchmark for quantitative evaluation of models to assess building damage using aerial videos. The second contribution is a new model, namely MSNet, which contains novel region proposal network designs and an unsupervi…
Timely disaster risk management requires accurate road maps and prompt damage assessment. Currently, this is done by volunteers manually marking satellite imagery of affected areas but this process is slow and often error-prone. Segmentation algorithms can be applied to satellite images to detect road networks. However, existing methods are unsuitable for disaster-struck areas as they make assumptions about the road network topology which may no longer be valid in these scenarios. Herein, we propose a CNN-based framework for identifying accessible roads in post-disaster imagery by detecting ch…
Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagery
Urban Resilience.AI Lab（城市韧性人工智能实验室）；Zachry Department of Civil and Environmental Engineering（扎克里土木与环境工程系）；Texas A&M University（德克萨斯A&M大学）；College Station, TX（学院站, 德克萨斯）
Carnegie Mellon University（卡内基梅隆大学）；Software Engineering Institute（软件工程研究所）；Defense Innovation Unit（国防创新部门）；Department of Defense（国防部）；CrowdAI, Inc.（CrowdAI公司）
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exists for using aerial imagery combined with computer vision algorithms to assess damage and reduce the potential danger to human life. In collaboration with multiple disaster response agencies, xBD prov…
Damage assessment after natural disasters is needed to distribute aid and forces to recovery from damage dealt optimally. This process involves acquiring satellite imagery for the region of interest, localization of buildings, and classification of the amount of damage caused by nature or urban factors to buildings. In case of natural disasters, this means processing many square kilometers of the area to judge whether a particular building had suffered from the damaging factors. In this work, we develop a computational approach for an automated comparison of the same region's satellite images …
Gaining timely and reliable situation awareness after hazard events such as a hurricane is crucial to emergency managers and first responders. One effective way to achieve that goal is through damage assessment. Recently, disaster researchers have been utilizing imagery captured through satellites or drones to quantify the number of flooded/damaged buildings. In this paper, we propose a mixed data approach, which leverages publicly available satellite imagery and geolocation features of the affected area to identify damaged buildings after a hurricane. The method demonstrated significant impro…
Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model
School of Computing, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院计算学院）；College of Business, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院商学院）；Graduate School of Data Science, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院数据科学研究生院）；Division of Social Science, Hong Kong University of Science and Technology (HKUST)（香港科技大学社会科学部）；Max Planck Institute for Security and Privacy (MPI-SP)（马克斯·普朗克安全与隐私研究所）
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Nanjing University of Science and Technology（南京理工大学）
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before an effective response is conducted. High-resolution satellite images provide rich information with pre- and post-disaster scenes for analysis. However, most existing works simply use pre- and post-disaster images as input without considering their correlations. In this paper, we propose a novel cross-directional fusion strategy to better explore the correlations between pre- and post-disaster images. Moreover, the da…
Humanitarian organizations must have fast and reliable data to respond to disasters. Deep learning approaches are difficult to implement in real-world disasters because it might be challenging to collect ground truth data of the damage situation (training data) soon after the event. The implementation of recent self-paced positive-unlabeled learning (PU) is demonstrated in this work by successfully applying to building damage assessment with very limited labeled data and a large amount of unlabeled data. Self-PU learning is compared with the supervised baselines and traditional PU learning usi…
Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning
Built Environment Department, College of Science and Technology, North Carolina A&T State University（北卡罗来纳农工州立大学科技学院建筑环境系）；United Nations University Institute for Water, Environment and Health（联合国大学水、环境与健康研究所）
Damage Assessment after Natural Disasters with UAVs: Semantic Feature Extraction using Deep Learning
University of Peradeniya（佩拉德尼亚大学）；Multidisciplinary AI Research Centre, University of Peradeniya（佩拉德尼亚大学多学科人工智能研究中心）；Vega Innovations（维加创新公司）；Queensland University of Technology (QUT)（昆士兰科技大学）
DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagery
organization= Department of Geography, Texas A\&M University , city= College Station , country= USA；organization= Department of Landscape Architecture \& Urban Planning, Texas A\&M University , city= College Station , country= USA；organization= Department of Industrial；Systems Engineering, University of Florida , city= Gainesville , country= USA；organization= Spatial Sciences Institute, University of Southern California , city= Los Angeles , country= USA；organization= Department of Geography；Sustainability, University of Tennessee , city= Knoxville , country= USA
Vulnerability and risk assessment of neighborhoods is essential for effective disaster preparedness. Existing traditional systems, due to dependency on time-consuming and cost-intensive field surveying, do not provide a scalable way to decipher warnings and assess the precise extent of the risk at a hyper-local level. In this work, machine learning was used to automate the process of identifying dwellings and their type to build a potentially more effective disaster vulnerability assessment system. First, satellite imageries of low-income settlements and vulnerable areas in India were used to …
Rapid post-disaster infrastructure damage characterisation enabled by remote sensing and deep learning technologies -- a tiered approach
University of Birmingham（伯明翰大学）；Centre for Research and Technology Hellas (CERTH)（希腊研究与技术中心）；Lviv National Polytechnic University（利沃夫国立理工大学）；University of Agriculture in Krakow（克拉科夫农业大学）；Brunel University London（伦敦布鲁内尔大学）
Improved MambdaBDA Framework for Robust Building Damage Assessment Across Disaster Domains
Department of Computer Engineering, Istanbul Technical University（计算机工程系，伊斯坦布尔技术大学）；Division of Engineering, New York University Abu Dhabi（工程学院，纽约大学阿布扎比分校）
Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles…
In this manuscript, an image analytics based deep learning framework for wind turbine blade surface damage detection is proposed. Turbine blade(s) which carry approximately one-third of a turbine weight are susceptible to damage and can cause sudden malfunction of a grid-connected wind energy conversion system. The surface damage detection of wind turbine blade requires a large dataset so as to detect a type of damage at an early stage. Turbine blade images are captured via aerial imagery. Upon inspection, it is found that the image dataset was limited and hence image augmentation is applied t…
