Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model
School of Computing, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院计算学院）；College of Business, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院商学院）；Graduate School of Data Science, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院数据科学研究生院）；Division of Social Science, Hong Kong University of Science and Technology (HKUST)（香港科技大学社会科学部）；Max Planck Institute for Security and Privacy (MPI-SP)（马克斯·普朗克安全与隐私研究所）
Kyeongjin Ahn, Sungwon Han, Sungwon Park, Jihee Kim, Sangyoon Park, Meeyoung Cha
The increasing frequency and intensity of natural disasters call for rapid and accurate damage assessment. In response, disaster benchmark datasets from high-resolution satellite imagery have been constructed to develop methods for detecting damaged areas. However, these methods face significant challenges when applied to previously unseen regions due to the limited geographical and disaster-type diversity in the existing datasets. We introduce DAVI (Disaster Assessment with VIsion foundation model), a novel approach that addresses domain disparities and detects structural damage at the building level without requiring ground-truth labels for target regions. DAVI combines task-specific knowledge from a model trained on source regions with task-agnostic knowledge from an image segmentation model to generate pseudo labels indicating potential damage in target regions. It then utilizes a two-stage refinement process, which operate at both pixel and image levels, to accurately identify changes in disaster-affected areas. Our evaluation, including a case study on the 2023 Türkiye earthquake, demonstrates that our model achieves exceptional performance across diverse terrains (e.g., North America, Asia, and the Middle East) and disaster types (e.g., wildfires, hurricanes, and tsunamis). This confirms its robustness in disaster assessment without dependence on ground-truth labels and highlights its practical applicability.
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）
The detrimental impacts of climate change include stronger and more destructive hurricanes happening all over the world. Identifying different damaged structures of an area including buildings and roads are vital since it helps the rescue team to plan their efforts to minimize the damage caused by a natural disaster. Semantic segmentation helps to identify different parts of an image. We implement a novel self-attention based semantic segmentation model on a high resolution UAV dataset and attain Mean IoU score of around 88% on the test set. The result inspires to use self-attention schemes in…
Massachusetts Institue of Technology（麻省理工学院）；WL Research
Automatic change detection and disaster damage assessment are currently procedures requiring a huge amount of labor and manual work by satellite imagery analysts. In the occurrences of natural disasters, timely change detection can save lives. In this work, we report findings on problem framing, data processing and training procedures which are specifically helpful for the task of building damage assessment using the newly released xBD dataset. Our insights lead to substantial improvement over the xBD baseline models, and we score among top results on the xView2 challenge leaderboard. We relea…
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（德克萨斯A&M大学）
In this paper, we present a large-scale hurricane Michael dataset for visual perception in disaster scenarios, and analyze state-of-the-art deep neural network models for semantic segmentation. The dataset consists of around 2000 high-resolution aerial images, with annotated ground-truth data for semantic segmentation. We discuss the challenges of the dataset and train the state-of-the-art methods on this dataset to evaluate how well these methods can recognize the disaster situations. Finally, we discuss challenges for future research.
In this paper, we study the problem of efficiently assessing building damage after natural disasters like hurricanes, floods or fires, through aerial video analysis. We make two main contributions. The first contribution is a new dataset, consisting of user-generated aerial videos from social media with annotations of instance-level building damage masks. This provides the first benchmark for quantitative evaluation of models to assess building damage using aerial videos. The second contribution is a new model, namely MSNet, which contains novel region proposal network designs and an unsupervi…
Carnegie Mellon University（卡内基梅隆大学）；Software Engineering Institute（软件工程研究所）；Defense Innovation Unit（国防创新部门）；Department of Defense（国防部）；CrowdAI, Inc.（CrowdAI公司）
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exists for using aerial imagery combined with computer vision algorithms to assess damage and reduce the potential danger to human life. In collaboration with multiple disaster response agencies, xBD prov…
In the aftermath of disasters, building damage maps are obtained using change detection to plan rescue operations. Current convolutional neural network approaches do not consider the similarities between neighboring buildings for predicting the damage. We present a novel graph-based building damage detection solution to capture these relationships. Our proposed model architecture learns from both local and neighborhood features to predict building damage. Specifically, we adopt the sample and aggregate graph convolution strategy to learn aggregation functions that generalize to unseen graphs w…
RIKEN Center for Advance Intelligence Project（理化学研究所革新智能研究中心）；the University of Tokyo（东京大学）；Hiroshima University（广岛大学）；Chiba University（千叶大学）；Tokyo Institute of Technology（东京工业大学）；Tohoku University（东北大学）
Earth observation technologies, such as optical imaging and synthetic aperture radar (SAR), provide excellent means to monitor ever-growing urban environments continuously. Notably, in the case of large-scale disasters (e.g., tsunamis and earthquakes), in which a response is highly time-critical, images from both data modalities can complement each other to accurately convey the full damage condition in the disaster's aftermath. However, due to several factors, such as weather and satellite coverage, it is often uncertain which data modality will be the first available for rapid disaster respo…
Automatic post-disaster damage detection using aerial imagery is crucial for quick assessment of damage caused by disaster and development of a recovery plan. The main problem preventing us from creating an applicable model in practice is that damaged (positive) examples we are trying to detect are much harder to obtain than undamaged (negative) examples, especially in short time. In this paper, we revisit the classical bootstrap aggregating approach in the context of modern transfer learning for data-efficient disaster damage detection. Unlike previous classical ensemble learning articles, ou…
Engineering Vellore Institute of Technology, AP Guntur, India（工程维洛雷理工学院，AP古尔图，印度）；Engineering Vellore Institute of Technology, AP Ongole, India（工程维洛雷理工学院，AP翁戈尔，印度）；Engineering Vellore Institute of Technology, AP Vijayawada, India（工程维洛雷理工学院，AP维杰亚瓦达，印度）
Rapid post-disaster infrastructure damage characterisation enabled by remote sensing and deep learning technologies -- a tiered approach
University of Birmingham（伯明翰大学）；Centre for Research and Technology Hellas (CERTH)（希腊研究与技术中心）；Lviv National Polytechnic University（利沃夫国立理工大学）；University of Agriculture in Krakow（克拉科夫农业大学）；Brunel University London（伦敦布鲁内尔大学）
University of Maryland Baltimore County（马里兰大学巴尔的摩县分校）；Texas A&M University（得克萨斯农工大学）
Visual scene understanding is the core task in making any crucial decision in any computer vision system. Although popular computer vision datasets like Cityscapes, MS-COCO, PASCAL provide good benchmarks for several tasks (e.g. image classification, segmentation, object detection), these datasets are hardly suitable for post disaster damage assessments. On the other hand, existing natural disaster datasets include mainly satellite imagery which have low spatial resolution and a high revisit period. Therefore, they do not have a scope to provide quick and efficient damage assessment tasks. Unm…
Accurate and fine-grained information about the extent of damage to buildings is essential for directing Humanitarian Aid and Disaster Response (HADR) operations in the immediate aftermath of any natural calamity. In recent years, satellite and UAV (drone) imagery has been used for this purpose, sometimes aided by computer vision algorithms. Existing Computer Vision approaches for building damage assessment typically rely on a two stage approach, consisting of building detection using an object detection model, followed by damage assessment through classification of the detected building tiles…
Timely disaster risk management requires accurate road maps and prompt damage assessment. Currently, this is done by volunteers manually marking satellite imagery of affected areas but this process is slow and often error-prone. Segmentation algorithms can be applied to satellite images to detect road networks. However, existing methods are unsuitable for disaster-struck areas as they make assumptions about the road network topology which may no longer be valid in these scenarios. Herein, we propose a CNN-based framework for identifying accessible roads in post-disaster imagery by detecting ch…
OPTNet: Ordering Point Transformer Network for Post-disaster 3D Semantic Segmentation
Computer Science and Engineering, Lehigh University, Bethlehem PA 18015, US（计算机科学与工程系，莱维大学，贝特莱姆 PA 18015，美国）；Civil and Environmental Engineering, Lehigh University, Bethlehem PA 18015, US（土木与环境工程系，莱维大学，贝特莱姆 PA 18015，美国）
Kansas State University（堪萨斯州立大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Traditional post-disaster assessment of damage heavily relies on expensive GIS data, especially remote sensing image data. In recent years, social media has become a rich source of disaster information that may be useful in assessing damage at a lower cost. Such information includes text (e.g., tweets) or images posted by eyewitnesses of a disaster. Most of the existing research explores the use of text in identifying situational awareness information useful for disaster response teams. The use of social media images to assess disaster damage is limited. In this paper, we propose a novel appro…
University of Maryland, Baltimore County（马里兰大学巴尔的摩县分校）
Visual Question Answering system integrated with Unmanned Aerial Vehicle (UAV) has a lot of potentials to advance the post-disaster damage assessment purpose. Providing assistance to affected areas is highly dependent on real-time data assessment and analysis. Scope of the Visual Question Answering is to understand the scene and provide query related answer which certainly faster the recovery process after any disaster. In this work, we address the importance of \textit{visual question answering (VQA)} task for post-disaster damage assessment by presenting our recently developed VQA dataset ca…
Hyperlocal disaster damage assessment using bi-temporal street-view imagery and pre-trained vision models
Department of Geography, Texas A&M University（地理系，德克萨斯A&M大学）；Department of Geography, Pennsylvania State University（地理系，宾夕法尼亚州立大学）；Department of Geography and Sustainability, University of Tennessee（地理与可持续发展系，田纳西大学）；Spatial Sciences Institute, University of Southern California（空间科学研究所，南加州大学）
The use of satellite imagery has become increasingly popular for disaster monitoring and response. After a disaster, it is important to prioritize rescue operations, disaster response and coordinate relief efforts. These have to be carried out in a fast and efficient manner since resources are often limited in disaster-affected areas and it's extremely important to identify the areas of maximum damage. However, most of the existing disaster mapping efforts are manual which is time-consuming and often leads to erroneous results. In order to address these issues, we propose a framework for chang…
We have developed a framework for crisis response and management that incorporates the latest technologies in computer vision (CV), inland flood prediction, damage assessment and data visualization. The framework uses data collected before, during, and after the crisis to enable rapid and informed decision making during all phases of disaster response. Our computer-vision model analyzes spaceborne and airborne imagery to detect relevant features during and after a natural disaster and creates metadata that is transformed into actionable information through web-accessible mapping tools. In part…
ChangeQuery: Advancing Remote Sensing Change Analysis for Natural and Human-Induced Disasters from Visual Detection to Semantic Understanding
School of Computer Science and Technology and the Ministry of Education Key Lab for Intelligent Networks and Network Security, Xi’an Jiaotong University（计算机科学与技术学院和教育部智能网络与网络安全重点实验室，西安交通大学）；State Key Laboratory of Remote Sensing and Digital Earth, Aerospace Information Research Institute, Chinese Academy of Sciences（遥感与数字地球国家重点实验室，航空信息研究所，中国科学院）；State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University（测绘遥感信息工程国家重点实验室，武汉大学）；Helmholtz-Zentrum Dresden-Rossendorf（德累斯顿-罗斯托克研究所）；Lancaster Environment Centre, Lancaster University（兰卡斯特大学环境研究中心）；Faculty of Electrical and Computer Engineering, University of Iceland（冰岛大学电气与计算机工程学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
Institute for Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））；Semnan University（塞姆南大学）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Social media plays a significant role in sharing essential information, which helps humanitarian organizations in rescue operations during and after disaster incidents. However, developing an efficient method that can provide rapid analysis of social media images in the early hours of disasters is still largely an open problem, mainly due to the lack of suitable datasets and the sheer complexity of this task. In addition, supervised methods can not generalize well to novel disaster incidents. In this paper, inspired by the success of Knowledge Distillation (KD) methods, we propose an unsupervi…
American University of Beirut（贝鲁特美国大学）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Change detection is instrumental to localize damage and understand destruction in disaster informatics. While convolutional neural networks are at the core of recent change detection solutions, we present in this work, BLDNet, a novel graph formulation for building damage change detection and enable learning relationships and representations from both local patterns and non-stationary neighborhoods. More specifically, we use graph convolutional networks to efficiently learn these features in a semi-supervised framework with few annotated data. Additionally, BLDNet formulation allows for the in…
DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagery
organization= Department of Geography, Texas A\&M University , city= College Station , country= USA；organization= Department of Landscape Architecture \& Urban Planning, Texas A\&M University , city= College Station , country= USA；organization= Department of Industrial；Systems Engineering, University of Florida , city= Gainesville , country= USA；organization= Spatial Sciences Institute, University of Southern California , city= Los Angeles , country= USA；organization= Department of Geography；Sustainability, University of Tennessee , city= Knoxville , country= USA
University of North Carolina at Charlotte（北卡罗来纳大学夏洛特分校）；Nanjing University of Science and Technology（南京理工大学）
Fast and effective responses are required when a natural disaster (e.g., earthquake, hurricane, etc.) strikes. Building damage assessment from satellite imagery is critical before an effective response is conducted. High-resolution satellite images provide rich information with pre- and post-disaster scenes for analysis. However, most existing works simply use pre- and post-disaster images as input without considering their correlations. In this paper, we propose a novel cross-directional fusion strategy to better explore the correlations between pre- and post-disaster images. Moreover, the da…
Improved MambdaBDA Framework for Robust Building Damage Assessment Across Disaster Domains
Department of Computer Engineering, Istanbul Technical University（计算机工程系，伊斯坦布尔技术大学）；Division of Engineering, New York University Abu Dhabi（工程学院，纽约大学阿布扎比分校）
Massachusetts Institute of Technology（麻省理工学院）；Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
Responding to natural disasters, such as earthquakes, floods, and wildfires, is a laborious task performed by on-the-ground emergency responders and analysts. Social media has emerged as a low-latency data source to quickly understand disaster situations. While most studies on social media are limited to text, images offer more information for understanding disaster and incident scenes. However, no large-scale image datasets for incident detection exists. In this work, we present the Incidents Dataset, which contains 446,684 images annotated by humans that cover 43 incidents across a variety o…
This study aims to enable more reliable automated post-disaster building damage classification using artificial intelligence (AI) and multi-view imagery. The current practices and research efforts in adopting AI for post-disaster damage assessment are generally (a) qualitative, lacking refined classification of building damage levels based on standard damage scales, and (b) trained based on aerial or satellite imagery with limited views, which, although indicative, are not completely descriptive of the damage scale. To enable more accurate and reliable automated quantification of damage levels…
After a disaster, teams of structural engineers collect vast amounts of images from damaged buildings to obtain lessons and gain knowledge from the event. Images of damaged buildings and components provide valuable evidence to understand the consequences on our structures. However, in many cases, images of damaged buildings are often captured without sufficient spatial context. Also, they may be hard to recognize in cases with severe damage. Incorporating past images showing a pre-disaster condition of such buildings is helpful to accurately evaluate possible circumstances related to a buildin…
Lancaster University（兰卡斯特大学）；University of Bath（巴斯大学）；University of Oxford（牛津大学）
High-resolution satellite imagery available immediately after disaster events is crucial for response planning as it facilitates broad situational awareness of critical infrastructure status such as building damage, flooding, and obstructions to access routes. Damage mapping at this scale would require hundreds of expert person-hours. However, a combination of crowdsourcing and recent advances in deep learning reduces the effort needed to just a few hours in real time. Asking volunteers to place point marks, as opposed to shapes of actual damaged areas, significantly decreases the required ana…
Simon Fraser University（西蒙弗雷泽大学）；Texas A&M University（德克萨斯A&M大学）
This paper presents \dahitra, a novel deep-learning model with hierarchical transformers to classify building damages based on satellite images in the aftermath of natural disasters. Satellite imagery provides real-time and high-coverage information and offers opportunities to inform large-scale post-disaster building damage assessment, which is critical for rapid emergency response. In this work, a novel transformer-based network is proposed for assessing building damage. This network leverages hierarchical spatial features of multiple resolutions and captures the temporal differences in the …
CSAIL at MIT（麻省理工学院计算机科学与人工智能实验室）；Technical University of Denmark (DTU)（丹麦技术大学）；Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；QCRI（卡塔尔计算研究所）
Natural disasters, such as floods, tornadoes, or wildfires, are increasingly pervasive as the Earth undergoes global warming. It is difficult to predict when and where an incident will occur, so timely emergency response is critical to saving the lives of those endangered by destructive events. Fortunately, technology can play a role in these situations. Social media posts can be used as a low-latency data source to understand the progression and aftermath of a disaster, yet parsing this data is tedious without automated methods. Prior work has mostly focused on text-based filtering, yet image…
Multi-Modal Attention for Automated Disaster Damage Assessment Using Remote Sensing Imagery and Deep Learning
Built Environment Department, College of Science and Technology, North Carolina A&T State University（北卡罗来纳农工州立大学科技学院建筑环境系）；United Nations University Institute for Water, Environment and Health（联合国大学水、环境与健康研究所）
3DAeroRelief: The first 3D Benchmark UAV Dataset for Post-Disaster Assessment
Department of Computer Science and Engineering, Lehigh University（计算机科学与工程系，莱维大学）；Department of Civil and Environmental Engineering, Lehigh University（土木与环境工程系，莱维大学）
The advancement of deep learning technology has enabled us to develop systems that outperform any other classification technique. However, success of any empirical system depends on the quality and diversity of the data available to train the proposed system. In this research, we have carefully accumulated a relatively challenging dataset that contains images collected from various sources for three different disasters: fire, water and land. Besides this, we have also collected images for various damaged infrastructure due to natural or man made calamities and damaged human due to war or accid…
Purdue University（普渡大学）；Lockheed Martin Space（洛克希德·马丁太空公司）
When disaster strikes, accurate situational information and a fast, effective response are critical to save lives. Widely available, high resolution satellite images enable emergency responders to estimate locations, causes, and severity of damage. Quickly and accurately analyzing the extensive amount of satellite imagery available, though, requires an automatic approach. In this paper, we present Siam-U-Net-Attn model - a multi-class deep learning model with an attention mechanism - to assess damage levels of buildings given a pair of satellite images depicting a scene before and after a disa…
