Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles. These multi-stage methods are not end-to-end trainable, and suffer from poor overall results. We propose RescueNet, a unified model that can simultaneously segment buildings and assess the damage levels to individual buildings and can be trained end-toend. In order to to model the composite nature of this problem, we propose a novel localization aware loss function, which consists of a Binary Cross Entropy loss for building segmentation, and a foreground only selective Categorical Cross-Entropy loss for damage classification, and show significant improvement over the widely used Cross-Entropy loss. RescueNet is tested on the large scale and diverse xBD dataset and achieves significantly better building segmentation and damage classification performance than previous methods and achieves generalization across varied geographical regions and disaster types.
Carnegie Mellon University（卡内基梅隆大学）；Software Engineering Institute（软件工程研究所）；Defense Innovation Unit（国防创新部门）；Department of Defense（国防部）；CrowdAI, Inc.（CrowdAI公司）
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exists for using aerial imagery combined with computer vision algorithms to assess damage and reduce the potential danger to human life. In collaboration with multiple disaster response agencies, xBD prov…
Massachusetts Institue of Technology（麻省理工学院）；WL Research
Automatic change detection and disaster damage assessment are currently procedures requiring a huge amount of labor and manual work by satellite imagery analysts. In the occurrences of natural disasters, timely change detection can save lives. In this work, we report findings on problem framing, data processing and training procedures which are specifically helpful for the task of building damage assessment using the newly released xBD dataset. Our insights lead to substantial improvement over the xBD baseline models, and we score among top results on the xView2 challenge leaderboard. We relea…
Puerto Rico suffered severe damage from the category 5 hurricane (Maria) in September 2017. Total monetary damages are estimated to be ~92 billion USD, the third most costly tropical cyclone in US history. The response to this damage has been tempered and slow moving, with recent estimates placing 45% of the population without power three months after the storm. Consequently, we developed a unique data-fusion mapping approach called the Urban Development Index (UDI) and new open source tool, Comet Time Series (CometTS), to analyze the recovery of electricity and infrastructure in Puerto Rico. …
New York University（纽约大学）；University of the Philippines Diliman（菲律宾大学迪里曼分校）；University of Washington（华盛顿大学）；German Corporation for International Cooperation GmbH (GIZ)（德国国际合作机构（GIZ））
Rapid damage assessment is of crucial importance to emergency responders during hurricane events, however, the evaluation process is often slow, labor-intensive, costly, and error-prone. New advances in computer vision and remote sensing open possibilities to observe the Earth at a different scale. However, substantial pre-processing work is still required in order to apply state-of-the-art methodology for emergency response. To enable the comparison of methods for automatic detection of damaged buildings from post-hurricane remote sensing imagery taken from both airborne and satellite sensors…
The usage of Unmanned Aerial Vehicles (UAVs) in the context of structural health inspection is recently gaining tremendous popularity. Camera mounted UAVs enable the fast acquisition of a large number of images often used for mapping, 3D model reconstruction, and as an assisting tool for inspectors. Due to the number of images captured during large scale UAV surveys, a manual image-based inspection analysis of entire assets cannot be efficiently performed by qualified engineers. Additionally, comparing defects to past inspections requires the retrieval of relevant images which is often impract…
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（德克萨斯A&M大学）
In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（得克萨斯农工大学）
Visual scene understanding is the core task in making any crucial decision in any computer vision system. Although popular computer vision datasets like Cityscapes, MS-COCO, PASCAL provide good benchmarks for several tasks (e.g. image classification, segmentation, object detection), these datasets are hardly suitable for post disaster damage assessments. On the other hand, existing natural disaster datasets include mainly satellite imagery which have low spatial resolution and a high revisit period. Therefore, they do not have a scope to provide quick and efficient damage assessment tasks. Unm…
We have developed a framework for crisis response and management that incorporates the latest technologies in computer vision (CV), inland flood prediction, damage assessment and data visualization. The framework uses data collected before, during, and after the crisis to enable rapid and informed decision making during all phases of disaster response. Our computer-vision model analyzes spaceborne and airborne imagery to detect relevant features during and after a natural disaster and creates metadata that is transformed into actionable information through web-accessible mapping tools. In part…
Timely disaster risk management requires accurate road maps and prompt damage assessment. Currently, this is done by volunteers manually marking satellite imagery of affected areas but this process is slow and often error-prone. Segmentation algorithms can be applied to satellite images to detect road networks. However, existing methods are unsuitable for disaster-struck areas as they make assumptions about the road network topology which may no longer be valid in these scenarios. Herein, we propose a CNN-based framework for identifying accessible roads in post-disaster imagery by detecting ch…
North Carolina State University（北卡罗来纳州立大学）；Colaberry Inc（Colaberry公司）；Iowa State University（爱荷华州立大学）
Severe weather events can cause large financial losses to farmers. Detailed information on the location and severity of damage will assist farmers, insurance companies, and disaster response agencies in making wise post-damage decisions. The goal of this study was a proof-of-concept to detect damaged crop areas from aerial imagery using computer vision and deep learning techniques. A specific objective was to compare existing object detection algorithms to determine which was best suited for crop damage detection. Two modes of crop damage common in maize (corn) production were simulated: stalk…
Thermal and RGB Images Work Better Together in Wind Turbine Damage Detection
Khmelnytskyi National University（赫梅利尼茨基国立大学）；West Ukrainian National University（西乌克兰国立大学）；Kazimierz Pulaski University of Technology and Humanities in Radom（拉多姆卡齐米日·普瓦斯基科技与人文大学）
Automatic post-disaster damage detection using aerial imagery is crucial for quick assessment of damage caused by disaster and development of a recovery plan. The main problem preventing us from creating an applicable model in practice is that damaged (positive) examples we are trying to detect are much harder to obtain than undamaged (negative) examples, especially in short time. In this paper, we revisit the classical bootstrap aggregating approach in the context of modern transfer learning for data-efficient disaster damage detection. Unlike previous classical ensemble learning articles, ou…
Enhancing deep learning performance on burned area delineation from SPOT-6/7 imagery for emergency management
IRISA, Université Bretagne Sud, UMR 6074, Vannes, France（IRISA，布列塔尼大学，UMR 6074，瓦纳兹，法国）；University of Salzburg, Department of Geoinformatics, Salzburg, Austria（萨尔茨堡大学，地理信息学院，萨尔茨堡，奥地利）；ICube-SERTIT, Université de Strasbourg, Illkirch Graffenstaden, France（ICube-SERTIT，斯特拉斯堡大学，伊尔克里希格拉芬施塔登，法国）
Open-Vocabulary vs Supervised Learning Methods for Post-Disaster Visual Scene Understanding
Department of Informatics and Telematics, Harokopio University of Athens（信息与电信学系，雅典哈罗科波斯大学）；Department of Networks and Digital Media, Kingston University（网络与数字媒体系，金士顿大学）；Department of Electrical and Computer Engineering, University of Western Macedonia（电子与计算机工程系，西马其顿大学）；Archimedes, Athena Research Center（阿基米德，雅典研究中心）
RIKEN Center for Advance Intelligence Project（理化学研究所革新智能研究中心）；the University of Tokyo（东京大学）；Hiroshima University（广岛大学）；Chiba University（千叶大学）；Tokyo Institute of Technology（东京工业大学）；Tohoku University（东北大学）
Earth observation technologies, such as optical imaging and synthetic aperture radar (SAR), provide excellent means to monitor ever-growing urban environments continuously. Notably, in the case of large-scale disasters (e.g., tsunamis and earthquakes), in which a response is highly time-critical, images from both data modalities can complement each other to accurately convey the full damage condition in the disaster's aftermath. However, due to several factors, such as weather and satellite coverage, it is often uncertain which data modality will be the first available for rapid disaster respo…
The use of satellite imagery has become increasingly popular for disaster monitoring and response. After a disaster, it is important to prioritize rescue operations, disaster response and coordinate relief efforts. These have to be carried out in a fast and efficient manner since resources are often limited in disaster-affected areas and it's extremely important to identify the areas of maximum damage. However, most of the existing disaster mapping efforts are manual which is time-consuming and often leads to erroneous results. In order to address these issues, we propose a framework for chang…
Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model
School of Computing, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院计算学院）；College of Business, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院商学院）；Graduate School of Data Science, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院数据科学研究生院）；Division of Social Science, Hong Kong University of Science and Technology (HKUST)（香港科技大学社会科学部）；Max Planck Institute for Security and Privacy (MPI-SP)（马克斯·普朗克安全与隐私研究所）
Gaining timely and reliable situation awareness after hazard events such as a hurricane is crucial to emergency managers and first responders. One effective way to achieve that goal is through damage assessment. Recently, disaster researchers have been utilizing imagery captured through satellites or drones to quantify the number of flooded/damaged buildings. In this paper, we propose a mixed data approach, which leverages publicly available satellite imagery and geolocation features of the affected area to identify damaged buildings after a hurricane. The method demonstrated significant impro…
Engineering Vellore Institute of Technology, AP Guntur, India（工程维洛雷理工学院，AP古尔图，印度）；Engineering Vellore Institute of Technology, AP Ongole, India（工程维洛雷理工学院，AP翁戈尔，印度）；Engineering Vellore Institute of Technology, AP Vijayawada, India（工程维洛雷理工学院，AP维杰亚瓦达，印度）
Massachusetts Institute of Technology（麻省理工学院）；MIT Lincoln Laboratory（麻省理工学院林肯实验室）
Aerial images provide important situational awareness for responding to natural disasters such as hurricanes. They are well-suited for providing information for damage estimation and localization (DEL); i.e., characterizing the type and spatial extent of damage following a disaster. Despite recent advances in sensing and unmanned aerial systems technology, much of post-disaster aerial imagery is still taken by handheld DSLR cameras from small, manned, fixed-wing aircraft. However, these handheld cameras lack IMU information, and images are taken opportunistically post-event by operators. As su…
After a hurricane, damage assessment is critical to emergency managers for efficient response and resource allocation. One way to gauge the damage extent is to quantify the number of flooded/damaged buildings, which is traditionally done by ground survey. This process can be labor-intensive and time-consuming. In this paper, we propose to improve the efficiency of building damage assessment by applying image classification algorithms to post-hurricane satellite imagery. At the known building coordinates (available from public data), we extract square-sized images from the satellite imagery to …
Google AI（谷歌人工智能）；UN World Food Programme（联合国世界粮食计划署）
In all types of disasters, from earthquakes to armed conflicts, aid workers need accurate and timely data such as damage to buildings and population displacement to mount an effective response. Remote sensing provides this data at an unprecedented scale, but extracting operationalizable information from satellite images is slow and labor-intensive. In this work, we use machine learning to automate the detection of building damage in satellite imagery. We compare the performance of four different convolutional neural network models in detecting damaged buildings in the 2010 Haiti earthquake. We…
Post-hurricane damage assessment is crucial towards managing resource allocations and executing an effective response. Traditionally, this evaluation is performed through field reconnaissance, which is slow, hazardous, and arduous. Instead, in this paper we furthered the idea of implementing deep learning through convolutional neural networks in order to classify post-hurricane satellite imagery of buildings as Flooded/Damaged or Undamaged. The experimentation was conducted employing a dataset containing post-hurricane satellite imagery from the Greater Houston area after Hurricane Harvey in 2…
In this manuscript, an image analytics based deep learning framework for wind turbine blade surface damage detection is proposed. Turbine blade(s) which carry approximately one-third of a turbine weight are susceptible to damage and can cause sudden malfunction of a grid-connected wind energy conversion system. The surface damage detection of wind turbine blade requires a large dataset so as to detect a type of damage at an early stage. Turbine blade images are captured via aerial imagery. Upon inspection, it is found that the image dataset was limited and hence image augmentation is applied t…
Purdue University（普渡大学）；Lockheed Martin Space（洛克希德·马丁太空公司）
When disaster strikes, accurate situational information and a fast, effective response are critical to save lives. Widely available, high resolution satellite images enable emergency responders to estimate locations, causes, and severity of damage. Quickly and accurately analyzing the extensive amount of satellite imagery available, though, requires an automatic approach. In this paper, we present Siam-U-Net-Attn model - a multi-class deep learning model with an attention mechanism - to assess damage levels of buildings given a pair of satellite images depicting a scene before and after a disa…
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Nanjing University of Science and Technology（南京理工大学）
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before an effective response is conducted. High-resolution satellite images provide rich information with pre- and post-disaster scenes for analysis. However, most existing works simply use pre- and post-disaster images as input without considering their correlations. In this paper, we propose a novel cross-directional fusion strategy to better explore the correlations between pre- and post-disaster images. Moreover, the da…
Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagery
Urban Resilience.AI Lab（城市韧性人工智能实验室）；Zachry Department of Civil and Environmental Engineering（扎克里土木与环境工程系）；Texas A&M University（德克萨斯A&M大学）；College Station, TX（学院站, 德克萨斯）
Skolkovo institute of Science and technology（斯科尔科沃科学技术研究院）；Moscow（莫斯科）
When major disaster occurs the questions are raised how to estimate the damage in time to support the decision making process and relief efforts by local authorities or humanitarian teams. In this paper we consider the use of Machine Learning and Computer Vision on remote sensing imagery to improve time efficiency of assessment of damaged buildings in disaster affected area. We propose a general workflow that can be useful in various disaster management applications, and demonstrate the use of the proposed workflow for the assessment of the damage caused by the wildfires in California in 2017.
Google Research（谷歌研究院）；Stanford University（斯坦福大学）；United Nations World Food Programme（联合国世界粮食计划署）
To respond to disasters such as earthquakes, wildfires, and armed conflicts, humanitarian organizations require accurate and timely data in the form of damage assessments, which indicate what buildings and population centers have been most affected. Recent research combines machine learning with remote sensing to automatically extract such information from satellite imagery, reducing manual labor and turn-around time. A major impediment to using machine learning methods in real disaster response scenarios is the difficulty of obtaining a sufficient amount of labeled data to train a model for a…
Multiclass Post-Earthquake Building Assessment Integrating High-Resolution Optical and SAR Satellite Imagery, Ground Motion, and Soil Data with Transformers
Department of Civil and Environmental Engineering, University of Houston, TX, USA（土木与环境工程系，德克萨斯大学）；Department of Electrical and Computer Engineering, University of Houston, TX, USA（电气与计算机工程系，德克萨斯大学）；Department of Earth and Atmospheric Science, University of Houston, TX, USA（地球与大气科学系，德克萨斯大学）；German Aerospace Center (DLR), Microwaves and Radar Institute, Munich, Germany（德国航天中心（DLR）微波与雷达研究所）
DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagery
organization= Department of Geography, Texas A\&M University , city= College Station , country= USA；organization= Department of Landscape Architecture \& Urban Planning, Texas A\&M University , city= College Station , country= USA；organization= Department of Industrial；Systems Engineering, University of Florida , city= Gainesville , country= USA；organization= Spatial Sciences Institute, University of Southern California , city= Los Angeles , country= USA；organization= Department of Geography；Sustainability, University of Tennessee , city= Knoxville , country= USA
Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning
Built Environment Department, College of Science and Technology, North Carolina A&T State University（北卡罗来纳农工州立大学科技学院建筑环境系）；United Nations University Institute for Water, Environment and Health（联合国大学水、环境与健康研究所）
Bridge Damage Detection from Low-Light UAV Imagery via Degradation-Aware Mixture-of-Experts Enhancement
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Sustainability X-Lab, The University of Hong Kong（香港大学可持续发展X实验室）；School of Architecture, Building, and Civil Engineering, Loughborough University（拉夫堡大学建筑、建造与土木工程学院）；Department of Engineering Science, University of Oxford（牛津大学工程科学系）；Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology（香港科技大学土木与环境工程系）
Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad bin Khalifa University（哈马德·本·哈利法大学）
We propose Nazr-CNN1, a deep learning pipeline for object detection and fine-grained classification in images acquired from Unmanned Aerial Vehicles (UAVs) for damage assessment and monitoring. Nazr-CNN consists of two components. The function of the first component is to localize objects (e.g. houses or infrastructure) in an image by carrying out a pixel-level classification. In the second component, a hidden layer of a Convolutional Neural Network (CNN) is used to encode Fisher Vectors (FV) of the segments generated from the first component in order to help discriminate between different lev…
