We have developed a framework for crisis response and management that incorporates the latest technologies in computer vision (CV), inland flood prediction, damage assessment and data visualization. The framework uses data collected before, during, and after the crisis to enable rapid and informed decision making during all phases of disaster response. Our computer-vision model analyzes spaceborne and airborne imagery to detect relevant features during and after a natural disaster and creates metadata that is transformed into actionable information through web-accessible mapping tools. In particular, we have designed an ensemble of models to identify features including water, roads, buildings, and vegetation from the imagery. We have investigated techniques to bootstrap and reduce dependency on large data annotation efforts by adding use of open source labels including OpenStreetMaps and adding complementary data sources including Height Above Nearest Drainage (HAND) as a side channel to the network's input to encourage it to learn other features orthogonal to visual characteristics. Modeling efforts include modification of connected U-Nets for (1) semantic segmentation, (2) flood line detection, and (3) for damage assessment. In particular for the case of damage assessment, we added a second encoder to U-Net so that it could learn pre-event and post-event image features simultaneously. Through this method, the network is able to learn the difference between the pre- and post-disaster images, and therefore more effectively classify the level of damage. We have validated our approaches using publicly available data from the National Oceanic and Atmospheric Administration (NOAA)'s Remote Sensing Division, which displays the city and street-level details as mosaic tile images as well as data released as part of the Xview2 challenge.
New York University（纽约大学）；University of the Philippines Diliman（菲律宾大学迪里曼分校）；University of Washington（华盛顿大学）；German Corporation for International Cooperation GmbH (GIZ)（德国国际合作机构（GIZ））
Rapid damage assessment is of crucial importance to emergency responders during hurricane events, however, the evaluation process is often slow, labor-intensive, costly, and error-prone. New advances in computer vision and remote sensing open possibilities to observe the Earth at a different scale. However, substantial pre-processing work is still required in order to apply state-of-the-art methodology for emergency response. To enable the comparison of methods for automatic detection of damaged buildings from post-hurricane remote sensing imagery taken from both airborne and satellite sensors…
In the United States, hurricanes are the most devastating natural disasters causing billions of dollars worth of damage every year. More importantly, construction jobsites are classified among the most vulnerable environments to severe wind events. During hurricanes, unsecured and incomplete elements of construction sites, such as scaffoldings, plywoods, and metal rods, will become the potential wind-borne debris, causing cascading damages to the construction projects and the neighboring communities. Thus, it is no wonder that construction firms implement jobsite emergency plans to enforce pre…
Gaining timely and reliable situation awareness after hazard events such as a hurricane is crucial to emergency managers and first responders. One effective way to achieve that goal is through damage assessment. Recently, disaster researchers have been utilizing imagery captured through satellites or drones to quantify the number of flooded/damaged buildings. In this paper, we propose a mixed data approach, which leverages publicly available satellite imagery and geolocation features of the affected area to identify damaged buildings after a hurricane. The method demonstrated significant impro…
After a hurricane, damage assessment is critical to emergency managers for efficient response and resource allocation. One way to gauge the damage extent is to quantify the number of flooded/damaged buildings, which is traditionally done by ground survey. This process can be labor-intensive and time-consuming. In this paper, we propose to improve the efficiency of building damage assessment by applying image classification algorithms to post-hurricane satellite imagery. At the known building coordinates (available from public data), we extract square-sized images from the satellite imagery to …
After a natural disaster, such as a hurricane, millions are left in need of emergency assistance. To allocate resources optimally, human planners need to accurately analyze data that can flow in large volumes from several sources. This motivates the development of multimodal machine learning frameworks that can integrate multiple data sources and leverage them efficiently. To date, the research community has mainly focused on unimodal reasoning to provide granular assessments of the damage. Moreover, previous studies mostly rely on post-disaster images, which may take several days to become av…
Physical media (like surveillance cameras) and social media (like Instagram and Twitter) may both be useful in attaining on-the-ground information during an emergency or disaster situation. However, the intersection and reliability of both surveillance cameras and social media during a natural disaster are not fully understood. To address this gap, we tested whether social media is of utility when physical surveillance cameras went off-line during Hurricane Irma in 2017. Specifically, we collected and compared geo-tagged Instagram and Twitter posts in the state of Florida during times and in a…
Massachusetts Institute of Technology（麻省理工学院）；Max Planck Institute for Biogeochemistry（马克斯·普朗克生物地球化学研究所）；Spatial Informatics Group（空间信息集团）；ENSTA Institut Polytechnique Paris（巴黎国立高等先进技术学院）；MIT Media Lab（麻省理工学院媒体实验室）；NASA Headquarters（美国国家航空航天局总部）；Latent Sciences（潜在科学公司）；University of Oxford（牛津大学）；Ubisoft（育碧）
As climate change increases the intensity of natural disasters, society needs better tools for adaptation. Floods, for example, are the most frequent natural disaster, but during hurricanes the area is largely covered by clouds and emergency managers must rely on nonintuitive flood visualizations for mission planning. To assist these emergency managers, we have created a deep learning pipeline that generates visual satellite images of current and future coastal flooding. We advanced a state-of-the-art GAN called pix2pixHD, such that it produces imagery that is physically-consistent with the ou…
Deep learning based landslide density estimation on SAR data for rapid response
University of California Berkeley（加州大学伯克利分校）；The Ohio State University（俄亥俄州立大学）；German Aerospace Center DLR（德国航空航天中心（DLR））；University of Valencia（瓦伦西亚大学）；National Observatory of Athens（雅典国家天文台）；University of Oxford（牛津大学）；NVIDIA（英伟达）；Universidad de Antioquia（安蒂奥基亚大学）
Simon Fraser University（西蒙弗雷泽大学）；Texas A&M University（德克萨斯A&M大学）
This paper presents \dahitra, a novel deep-learning model with hierarchical transformers to classify building damages based on satellite images in the aftermath of natural disasters. Satellite imagery provides real-time and high-coverage information and offers opportunities to inform large-scale post-disaster building damage assessment, which is critical for rapid emergency response. In this work, a novel transformer-based network is proposed for assessing building damage. This network leverages hierarchical spatial features of multiple resolutions and captures the temporal differences in the …
3DAeroRelief: The first 3D Benchmark UAV Dataset for Post-Disaster Assessment
Department of Computer Science and Engineering, Lehigh University（计算机科学与工程系，莱维大学）；Department of Civil and Environmental Engineering, Lehigh University（土木与环境工程系，莱维大学）
Recov-Vision: Linking Street View Imagery and Vision-Language Models for Post-Disaster Recovery
UrbanResilience.AI Lab（UrbanResilience.AI 实验室）；Zachry Department of Civil and Environmental Engineering（Zachry 土木与环境工程系）；Texas A&M University（德克萨斯A&M大学）；Department of Earth, Marine and Environmental Sciences（地球、海洋与环境科学系）；The University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
