Detecting Damage Building Using Real-time Crowdsourced Images and Transfer Learning
Zymergen（Zymergen公司）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；Berkeley Seismological Laboratory（伯克利地震实验室）；AT&T（美国电话电报公司）；University of California, Berkeley（加州大学伯克利分校）
Gaurav Chachra, Qingkai Kong, Jim Huang, Srujay Korlakunta, Jennifer Grannen, Alexander Robson, Richard Allen
After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged building images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important locations on the images that facilitate the decision.
Lancaster University（兰卡斯特大学）；University of Bath（巴斯大学）；University of Oxford（牛津大学）
High-resolution satellite imagery available immediately after disaster events is crucial for response planning as it facilitates broad situational awareness of critical infrastructure status such as building damage, flooding, and obstructions to access routes. Damage mapping at this scale would require hundreds of expert person-hours. However, a combination of crowdsourcing and recent advances in deep learning reduces the effort needed to just a few hours in real time. Asking volunteers to place point marks, as opposed to shapes of actual damaged areas, significantly decreases the required ana…
Open-access model for detecting openly dumped dispersed municipal solid waste from crowdsourced UAV imagery in Sub-Saharan Africa
Heidelberg University（海德堡大学）；Global Facility for Disaster Reduction and Recovery (GFDRR), World Bank（世界银行全球减灾与恢复基金（GFDRR））；The University of Chicago（芝加哥大学）；Santa Fe Institute（圣塔菲研究所）；Complexity Science Hub（维也纳复杂科学中心）；Imperial College London（帝国理工学院）；Humanitarian OpenStreetMap Team（人道开放街图团队）；Open Map Development Tanzania（坦桑尼亚开放地图发展组织）；Data4Moz（莫桑比克数据组织）
Wind energy's ability to compete with fossil fuels on a market level depends on lowering wind's high operational costs. Since damages on wind turbine blades are the leading cause for these operational problems, identifying blade damages is critical. However, recent works in visual identification of blade damages are still experimental and focus on optimizing the traditional machine learning metrics such as IoU. In this paper, we argue that pushing models to production long before achieving the "optimal" model performance can still generate real value for this use case. We discuss the performan…
Massachusetts Institute of Technology（麻省理工学院）；Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）
Responding to natural disasters, such as earthquakes, floods, and wildfires, is a laborious task performed by on-the-ground emergency responders and analysts. Social media has emerged as a low-latency data source to quickly understand disaster situations. While most studies on social media are limited to text, images offer more information for understanding disaster and incident scenes. However, no large-scale image datasets for incident detection exists. In this work, we present the Incidents Dataset, which contains 446,684 images annotated by humans that cover 43 incidents across a variety o…
FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence
College of Computing and Data Science, Nanyang Technological University（南洋理工大学计算机与数据科学学院）；Alibaba-NTU Global e-Sustainability CorpLab (ANGEL)（阿里巴巴-NTU全球可持续性科技实验室）；Alibaba Group（阿里巴巴集团）
Rapid post-disaster infrastructure damage characterisation enabled by remote sensing and deep learning technologies -- a tiered approach
University of Birmingham（伯明翰大学）；Centre for Research and Technology Hellas (CERTH)（希腊研究与技术中心）；Lviv National Polytechnic University（利沃夫国立理工大学）；University of Agriculture in Krakow（克拉科夫农业大学）；Brunel University London（伦敦布鲁内尔大学）
Google AI（谷歌人工智能）；UN World Food Programme（联合国世界粮食计划署）
In all types of disasters, from earthquakes to armed conflicts, aid workers need accurate and timely data such as damage to buildings and population displacement to mount an effective response. Remote sensing provides this data at an unprecedented scale, but extracting operationalizable information from satellite images is slow and labor-intensive. In this work, we use machine learning to automate the detection of building damage in satellite imagery. We compare the performance of four different convolutional neural network models in detecting damaged buildings in the 2010 Haiti earthquake. We…
Indian Institute of Technology Roorkee（印度理工学院鲁尔基分校）；University of Tokyo（东京大学）；Amazon EU（亚马逊欧盟）
Many municipalities and road authorities seek to implement automated evaluation of road damage. However, they often lack technology, know-how, and funds to afford state-of-the-art equipment for data collection and analysis of road damages. Although some countries, like Japan, have developed less expensive and readily available Smartphone-based methods for automatic road condition monitoring, other countries still struggle to find efficient solutions. This work makes the following contributions in this context. Firstly, it assesses the usability of the Japanese model for other countries. Second…
Image segmentation is the most challenging issue in computer vision applications. And most difficulties for crops management in agriculture are the lack of appropriate methods for detecting the leaf damage for pests treatment. In this paper we proposed an automatic method for leaf damage detection and severity estimation of coffee leaf by avoiding defoliation. After enhancing the contrast of the original image using LUT based gamma correction, the image is processed to remove the background, and the output leaf is clustered using Fuzzy c-means segmentation in V channel of YUV color space to ma…
North Carolina State University（北卡罗来纳州立大学）；Colaberry Inc（Colaberry公司）；Iowa State University（爱荷华州立大学）
Severe weather events can cause large financial losses to farmers. Detailed information on the location and severity of damage will assist farmers, insurance companies, and disaster response agencies in making wise post-damage decisions. The goal of this study was a proof-of-concept to detect damaged crop areas from aerial imagery using computer vision and deep learning techniques. A specific objective was to compare existing object detection algorithms to determine which was best suited for crop damage detection. Two modes of crop damage common in maize (corn) production were simulated: stalk…
Sam Houston State University（萨姆休斯顿州立大学）；Academy of Finance（金融学院）；FPT University（FPT大学）
Maintaining roadway infrastructure is essential for ensuring a safe, efficient, and sustainable transportation system. However, manual data collection for detecting road damage is time-consuming, labor-intensive, and poses safety risks. Recent advancements in artificial intelligence, particularly deep learning, offer a promising solution for automating this process using road images. This paper presents a comprehensive workflow for road damage detection using deep learning models, focusing on optimizations for inference speed while preserving detection accuracy. Specifically, to accommodate ha…
Norwegian University of Science and Technology（挪威科技大学）；Wuhan University（武汉大学）；Nanjing University of Finance & Economics（南京财经大学）
Navigation services utilized by autonomous vehicles or ordinary users require the availability of detailed information about road-related objects and their geolocations, especially at road intersections. However, these road intersections are mainly represented as point elements without detailed information, or are even not available in current versions of crowdsourced mapping databases including OpenStreetMap(OSM). This study develops an approach to automatically detect road objects and place them to right location from street-level images. Our processing pipeline relies on two convolutional n…
Institute for Research in Fundamental Sciences (IPM)（基础科学研究所（IPM））；Semnan University（塞姆南大学）；Hamad Bin Khalifa University（哈马德·本·哈利法大学）
Social media plays a significant role in sharing essential information, which helps humanitarian organizations in rescue operations during and after disaster incidents. However, developing an efficient method that can provide rapid analysis of social media images in the early hours of disasters is still largely an open problem, mainly due to the lack of suitable datasets and the sheer complexity of this task. In addition, supervised methods can not generalize well to novel disaster incidents. In this paper, inspired by the success of Knowledge Distillation (KD) methods, we propose an unsupervi…
Massachusetts Institute of Technology（麻省理工学院）；MIT Lincoln Laboratory（麻省理工学院林肯实验室）
Aerial images provide important situational awareness for responding to natural disasters such as hurricanes. They are well-suited for providing information for damage estimation and localization (DEL); i.e., characterizing the type and spatial extent of damage following a disaster. Despite recent advances in sensing and unmanned aerial systems technology, much of post-disaster aerial imagery is still taken by handheld DSLR cameras from small, manned, fixed-wing aircraft. However, these handheld cameras lack IMU information, and images are taken opportunistically post-event by operators. As su…
Plant parasitic nematodes cause damage to crop plants on a global scale. Robust detection on image data is a prerequisite for monitoring such nematodes, as well as for many biological studies involving the nematode C. elegans, a common model organism. Here, we propose a framework for detecting worm-shaped objects in microscopic images that is based on convolutional neural networks (CNNs). We annotate nematodes with curved lines along the body, which is more suitable for worm-shaped objects than bounding boxes. The trained model predicts worm skeletons and body endpoints. The endpoints serve to…
Qatar Computing Research Institute, HBKU（哈马德·本·哈利法大学卡塔尔计算研究所）；BJIT Limited（BJIT有限公司）
During a disaster event, images shared on social media helps crisis managers gain situational awareness and assess incurred damages, among other response tasks. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of damage. Despite several efforts, past works mainly suffer from limited resources (i.e., labeled images) available to train more…
A Deep Learning framework for building damage assessment using VHR SAR and geospatial data: demonstration on the 2023 Turkiye Earthquake
Department of Electrical, Computer and Biomedical Engineering, University of Pavia（帕维亚大学电气、计算机与生物医学工程系）；Department of Engineering, University of Sannio（萨诺尼奥大学工程系）；Italian Space Agency (ASI)（意大利航天局）
National Yang Ming Chiao Tung University（国立阳明交通大学）；National Cheng Kung University Hospital（国立成功大学医院）；National Cheng Kung University（国立成功大学）
The clinical symptoms of pulmonary embolism (PE) are very diverse and non-specific, which makes it difficult to diagnose. In addition, pulmonary embolism has multiple triggers and is one of the major causes of vascular death. Therefore, if it can be detected and treated quickly, it can significantly reduce the risk of death in hospitalized patients. In the detection process, the cost of computed tomography pulmonary angiography (CTPA) is high, and angiography requires the injection of contrast agents, which increase the risk of damage to the patient. Therefore, this study will use a deep learn…
COMSATS University Islamabad（伊斯兰堡COMSATS大学）；Bacha Khan University（巴查汗大学）；Chung-Ang University（中央大学）；University of Malaya（马来亚大学）；University Malaysia Sabah（马来西亚沙巴大学）；Institute of Space Technology（空间技术研究所）
The ongoing COVID-19 pandemic has already taken millions of lives and damaged economies across the globe. Most COVID-19 deaths and economic losses are reported from densely crowded cities. It is comprehensible that the effective control and prevention of epidemic/pandemic infectious diseases is vital. According to WHO, testing and diagnosis is the best strategy to control pandemics. Scientists worldwide are attempting to develop various innovative and cost-efficient methods to speed up the testing process. This paper comprehensively evaluates the applicability of the recent top ten state-of-th…
Silesian University of Technology（西里西亚工业大学）；Polish Academy of Sciences（波兰科学院）
The area affected by the earthquake is vast and often difficult to entirely cover, and the earthquake itself is a sudden event that causes multiple defects simultaneously, that cannot be effectively traced using traditional, manual methods. This article presents an innovative approach to the problem of detecting damage after sudden events by using an interconnected set of deep machine learning models organized in a single pipeline and allowing for easy modification and swapping models seamlessly. Models in the pipeline were trained with a synthetic dataset and were adapted to be further evalua…
FaraAI Company（法拉人工智能公司）；University of Tehran（德黑兰大学）
Optical coherence tomography (OCT) imaging is a well-known technology for visualizing retinal layers and helps ophthalmologists to detect possible diseases. Accurate and early diagnosis of common retinal diseases can prevent the patients from suffering critical damages to their vision. Computer-aided diagnosis (CAD) systems can significantly assist ophthalmologists in improving their examinations. This paper presents a new enhanced deep ensemble convolutional neural network for detecting retinal diseases from OCT images. Our model generates rich and multi-resolution features by employing the l…
A Benchmark & Dataset for Detecting AI-Manipulated Visual Evidence in the Court System
University of Toronto（多伦多大学）；University of Waterloo（滑铁卢大学）；University of British Columbia（不列颠哥伦比亚大学）；University of Western Ontario（西安大略大学）；University of Ottawa（渥太华大学）
Qatar Computing Research Institute, HBKU（哈马德·本·卡利法大学卡塔尔计算研究所）；BJIT Limited（BJIT有限公司）
Images shared on social media help crisis managers gain situational awareness and assess incurred damages, among other response tasks. As the volume and velocity of such content are typically high, real-time image classification has become an urgent need for a faster disaster response. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of t…
