Flood Detection via Twitter Streams using Textual and Visual Features

Floods Detection in Twitter Text and Images

In this paper, we present our methods for the MediaEval 2020 Flood Related Multimedia task, which aims to analyze and combine textual and visual content from social media for the detection of real-world flooding events. The task mainly focuses on identifying floods related tweets relevant to a specific area. We propose several schemes to address the challenge. For text-based flood events detection, we use three different methods, relying on Bog of Words (BOW) and an Italian Version of Bert individually and in combination, achieving an F1-score of 0.77%, 0.68%, and 0.70% on the development set,…

Finding Relevant Flood Images on Twitter using Content-based Filters

The analysis of natural disasters such as floods in a timely manner often suffers from limited data due to coarsely distributed sensors or sensor failures. At the same time, a plethora of information is buried in an abundance of images of the event posted on social media platforms such as Twitter. These images could be used to document and rapidly assess the situation and derive proxy-data not available from sensors, e.g., the degree of water pollution. However, not all images posted online are suitable or informative enough for this purpose. Therefore, we propose an automatic filtering approa…

一种面向可持续城市洪水预测的具有频率感知通道与空间细化的可解释深度神经网络

An Explainable Deep Neural Network with Frequency-Aware Channel and Spatial Refinement for Flood Prediction in Sustainable Cities

针对城市洪水预测，提出XFloodNet框架，集成层次化跨模态门控注意力、异构卷积自适应多尺度注意力和级联卷积Transformer细化，在三个基准数据集上取得最优F1分数。

Multi-Modal Machine Learning for Flood Detection in News, Social Media and Satellite Sequences

In this paper we present our methods for the MediaEval 2019 Mul-timedia Satellite Task, which is aiming to extract complementaryinformation associated with adverse events from Social Media andsatellites. For the first challenge, we propose a framework jointly uti-lizing colour, object and scene-level information to predict whetherthe topic of an article containing an image is a flood event or not.Visual features are combined using early and late fusion techniquesachieving an average F1-score of82.63,82.40,81.40and76.77. Forthe multi-modal flood level estimation, we rely on both visualand textu…

Aiding Intra-Text Representations with Visual Context for Multimodal Named Entity Recognition

With massive explosion of social media such as Twitter and Instagram, people daily share billions of multimedia posts, containing images and text. Typically, text in these posts is short, informal and noisy, leading to ambiguities which can be resolved using images. In this paper we explore text-centric Named Entity Recognition task on these multimedia posts. We propose an end to end model which learns a joint representation of a text and an image. Our model extends multi-dimensional self attention technique, where now image helps to enhance relationship between words. Experiments show that ou…

群体的情绪：通过跨模态蒸馏从推文学习图像情感

The Emotions of the Crowd: Learning Image Sentiment from Tweets via Cross-modal Distillation

针对社交媒体图像情感极性预测依赖人工标注训练集的问题，提出基于跨模态蒸馏的自动化方法，用文本教师模型指导视觉学生模型训练，构建约150万张的弱标注数据集，在五个基准上超越当前最优水平。