Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detection
Institute of Advanced Research in Artificial Intelligence（人工智能高级研究院）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗赖贝格资源技术研究所）
Omid Ghorbanzadeh, Yonghao Xu, Pedram Ghamisi, Michael Kopp, David Kreil
发表信息：
IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-17, 2022
This study introduces \textit{Landslide4Sense}, a reference benchmark for landslide detection from remote sensing. The repository features 3,799 image patches fusing optical layers from Sentinel-2 sensors with the digital elevation model and slope layer derived from ALOS PALSAR. The added topographical information facilitates the accurate detection of landslide borders, which recent researches have shown to be challenging using optical data alone. The extensive data set supports deep learning (DL) studies in landslide detection and the development and validation of methods for the systematic update of landslide inventories. The benchmark data set has been collected at four different times and geographical locations: Iburi (September 2018), Kodagu (August 2018), Gorkha (April 2015), and Taiwan (August 2009). Each image pixel is labelled as belonging to a landslide or not, incorporating various sources and thorough manual annotation. We then evaluate the landslide detection performance of 11 state-of-the-art DL segmentation models: U-Net, ResU-Net, PSPNet, ContextNet, DeepLab-v2, DeepLab-v3+, FCN-8s, LinkNet, FRRN-A, FRRN-B, and SQNet. All models were trained from scratch on patches from one quarter of each study area and tested on independent patches from the other three quarters. Our experiments demonstrate that ResU-Net outperformed the other models for the landslide detection task. We make the multi-source landslide benchmark data (Landslide4Sense) and the tested DL models publicly available at \url{https://www.iarai.ac.at/landslide4sense}, establishing an important resource for remote sensing, computer vision, and machine learning communities in studies of image classification in general and applications to landslide detection in particular.
LMHLD: A Large-scale Multi-source High-resolution Landslide Dataset for Landslide Detection based on Deep Learning
School of Future Technology, China University of Geosciences（中国地质大学未来技术学院）；School of Geophysics and Geomatics, China University of Geosciences（中国地质大学地球物理与空间信息学院）；Aerospace Information Research Institute, Henan Academy of Science（河南省科学院航天信息研究所）
RMAU-NET: A Residual-Multihead-Attention U-Net Architecture for Landslide Segmentation and Detection from Remote Sensing Images
Van Lang University（范兰大学）；Austrian Institute of Technology（奥地利技术研究院）；Troyes University（特鲁瓦大学）；HCM University of Technology（胡志明技术大学）；Ton Duc Thang University（-ton Duc Thang大学）
本文提出了一种基于残差多头注意力 U-Net 架构的端到端深度学习模型 RMAU-NET，利用遥感图像进行滑坡检测与分割，在多个基准数据集上取得了优异的 F1 和 mIoU 分数，证明了其在实际滑坡观测系统中的应用潜力。
EarthLD: Towards Unified Open-World Landslide Understanding via Vision-Language Guided Diffusion Models
University of Macau（澳门大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；Xi’an University of Science and Technology（西安科技大学）
Global-Local Contextual Progressive Expansion Network for Martian Landslide Segmentation in Multimodal Remote Sensing Imagery
Computer Vision Center, Universitat Autònoma de Barcelona（巴塞罗那自治大学计算机视觉中心）；Fairfield University（费尔菲尔德大学）；Michigan Technological University（密歇根理工大学）；University of Kerala（喀拉拉大学）；ESPOL Polytechnic University（ESPOL理工大学）；University of Mississippi（密西西比大学）
Institute of Advanced Research in Artificial Intelligence (IARAI)（高级人工智能研究所（IARAI））；Wuhan University（武汉大学）；Xidian University（西安电子科技大学）；Technical University of Munich（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心（DLR））；Chengdu University of Technology（成都理工大学）
The scientific outcomes of the 2022 Landslide4Sense (L4S) competition organized by the Institute of Advanced Research in Artificial Intelligence (IARAI) are presented here. The objective of the competition is to automatically detect landslides based on large-scale multiple sources of satellite imagery collected globally. The 2022 L4S aims to foster interdisciplinary research on recent developments in deep learning (DL) models for the semantic segmentation task using satellite imagery. In the past few years, DL-based models have achieved performance that meets expectations on image interpretati…
Frame Evidence:A Benchmark for Reference Frames and Visual Evidence in Multi-Image Spatial Reasoning
Shanghai University（上海大学）；East China University of Science and Technology（华东理工大学）；ShanghaiTech University（上海科技大学）；Southwest University of Science and Technology（西南科技大学）；Shanghai Ocean University（上海海洋大学）；Nanjing University of Science and Technology（南京理工大学）
Universidad de Buenos Aires（布宜诺斯艾利斯大学）；CONICET（阿根廷国家科学与技术研究理事会）
Visual search is an essential part of almost any everyday human goal-directed interaction with the environment. Nowadays, several algorithms are able to predict gaze positions during simple observation, but few models attempt to simulate human behavior during visual search in natural scenes. Furthermore, these models vary widely in their design and exhibit differences in the datasets and metrics with which they were evaluated. Thus, there is a need for a reference point, on which each model can be tested and from where potential improvements can be derived. In the present work, we select publi…
Image copy detection (ICD) aims to determine whether a query image is an edited copy of any image from a reference set. Currently, there are very limited public benchmarks for ICD, while all overlook a critical challenge in real-world applications, i.e., the distraction from hard negative queries. Specifically, some queries are not edited copies but are inherently similar to some reference images. These hard negative queries are easily false recognized as edited copies, significantly compromising the ICD accuracy. This observation motivates us to build the first ICD benchmark featuring this ch…
University of Sao Paulo（圣保罗大学）；SKLOIS, IIE, CAS（中国科学院信息工程研究所信息安全国家重点实验室）；Texas A&M University（得克萨斯农工大学）；Tianjin University（天津大学）
We present a comprehensive study and evaluation of existing single image deraining algorithms, using a new large-scale benchmark consisting of both synthetic and real-world rainy images.This dataset highlights diverse data sources and image contents, and is divided into three subsets (rain streak, rain drop, rain and mist), each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments …
MIRROR: Manifold Ideal Reference ReconstructOR for Generalizable AI-Generated Image Detection
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Advanced Interdisciplinary Sciences, UCAS（北京大学交叉信息学院）；Huazhong University of Science and Technology（华中科技大学）；Tencent YouTu Lab（腾讯YouTu实验室）；Southwest University（西南大学）；Peking University（北京大学）；The University of Sydney（悉尼大学）；Shenzhen University（深圳大学）；Tsinghua University（清华大学）
UC Santa Barbara（圣巴巴拉大学）；UC Santa Cruz（圣克鲁斯大学）；Google（谷歌公司）
A major challenge in visually grounded language generation is to build robust benchmark datasets and models that can generalize well in real-world settings. To do this, it is critical to ensure that our evaluation protocols are correct, and benchmarks are reliable. In this work, we set forth to design a set of experiments to understand an important but often ignored problem in visually grounded language generation: given that humans have different utilities and visual attention, how will the sample variance in multi-reference datasets affect the models' performance? Empirically, we study sever…
Thapar Institute of Engineering and Technology（塔帕尔工程技术学院）；Oxford Brookes University（牛津布鲁克斯大学）
With the inclusion of camera in daily life, an automatic no reference image quality evaluation index is required for automatic classification of images. The present manuscripts proposes a new No Reference Regional Mutual Information based technique for evaluating the quality of an image. We use regional mutual information on subsets of the complete image. Proposed technique is tested on four benchmark natural image databases, and one benchmark synthetic database. A comparative analysis with classical and state-of-art methods indicate superiority of the present technique for high quality images…
The main goal of objective image quality assessment is to devise computational, mathematical models which are able to predict perceptual image quality consistently with subjective evaluations. The evaluation of objective image quality assessment algorithms is based on experiments conducted on publicly available benchmark databases. In this study, our goal is to give a comprehensive evaluation about no-reference image quality assessment algorithms, whose original source codes are available online, using the recently published KADID-10k database which is one of the largest available benchmark da…
IOI: Invisible One-Iteration Adversarial Attack on No-Reference Image- and Video-Quality Metrics
Lomonosov Moscow State University（莫斯科罗蒙诺索夫国立大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息学研究所可信人工智能研究中心）；MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）
We present a comprehensive study and evaluation of existing single image compression artifacts removal algorithms, using a new 4K resolution benchmark including diversified foreground objects and background scenes with rich structures, called Large-scale Ideal Ultra high definition 4K (LIU4K) benchmark. Compression artifacts removal, as a common post-processing technique, aims at alleviating undesirable artifacts such as blockiness, ringing, and banding caused by quantization and approximation in the compression process. In this work, a systematic listing of the reviewed methods is presented b…
The goal of full-reference image quality assessment (FR-IQA) is to predict the quality of an image as perceived by human observers with using its pristine, reference counterpart. In this study, we explore a novel, combined approach which predicts the perceptual quality of a distorted image by compiling a feature vector from convolutional activation maps. More specifically, a reference-distorted image pair is run through a pretrained convolutional neural network and the activation maps are compared with a traditional image similarity metric. Subsequently, the resulted feature vector is mapped o…
Multi-level deep-features have been driving state-of-the-art methods for aesthetics and image quality assessment (IQA). However, most IQA benchmarks are comprised of artificially distorted images, for which features derived from ImageNet under-perform. We propose a new IQA dataset and a weakly supervised feature learning approach to train features more suitable for IQA of artificially distorted images. The dataset, KADIS-700k, is far more extensive than similar works, consisting of 140,000 pristine images, 25 distortions types, totaling 700k distorted versions. Our weakly supervised feature le…
CogCanvas: A Benchmark for Evaluating Multi-Subject Reference-Based Image Generation
University of Science, Ho Chi Minh City, Vietnam（胡志明市理科大学）；University of Dayton, Ohio, United States（代顿大学）；Vietnam National University, Ho Chi Minh City, Vietnam（越南国家大学胡志明市分校）
StructGen: Disambiguating Multi-Reference Image Generation via Structured Context Modeling
Institute of Information Science, Beijing Jiaotong University（北京交通大学信息科学研究所）；Institute of Big Data, Fudan University（复旦大学大数据研究院）；Visual Intelligence + X International Joint Laboratory of the Ministry of Education（教育部视觉智能+X国际联合实验室）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；MT Lab, Meitu Inc（美图公司MT实验室）
Zhejiang University（浙江大学）；Columbia University（哥伦比亚大学）；Hong Kong University of Science and Technology（香港科技大学）
Distinctive Image Captioning (DIC) -- generating distinctive captions that describe the unique details of a target image -- has received considerable attention over the last few years. A recent DIC work proposes to generate distinctive captions by comparing the target image with a set of semantic-similar reference images, i.e., reference-based DIC (Ref-DIC). It aims to make the generated captions can tell apart the target and reference images. Unfortunately, reference images used by existing Ref-DIC works are easy to distinguish: these reference images only resemble the target image at scene-l…
Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks
MSU Institute for Artificial Intelligence（莫斯科国立大学人工智能研究所）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院系统编程研究所可信人工智能研究中心）；Lomonosov Moscow State University（罗蒙诺索夫莫斯科国立大学）
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
A video-grounded dialogue system referred to as the Structured Co-reference Graph Attention (SCGA) is presented for decoding the answer sequence to a question regarding a given video while keeping track of the dialogue context. Although recent efforts have made great strides in improving the quality of the response, performance is still far from satisfactory. The two main challenging issues are as follows: (1) how to deduce co-reference among multiple modalities and (2) how to reason on the rich underlying semantic structure of video with complex spatial and temporal dynamics. To this end, SCG…
MultiBind: A Benchmark for Attribute Misbinding in Multi-Subject Generation
New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；The University of Chicago（芝加哥大学）；ByteDance（字节跳动）
Tel Aviv University（特拉维夫大学）；Information Sciences Institute USC（南加州大学信息科学研究所）；The Open University of Israel（以色列开放大学）；Facebook AI Research（Facebook人工智能研究院）
We present a general approach to video understanding, inspired by semantic transfer techniques that have been successfully used for 2D image analysis. Our method considers a video to be a 1D sequence of clips, each one associated with its own semantics. The nature of these semantics -- natural language captions or other labels -- depends on the task at hand. A test video is processed by forming correspondences between its clips and the clips of reference videos with known semantics, following which, reference semantics can be transferred to the test video. We describe two matching methods, bot…
Computer Vision Center（计算机视觉中心）；University of Florence（佛罗伦萨大学）
We propose a no-reference image quality assessment (NR-IQA) approach that learns from rankings (RankIQA). To address the problem of limited IQA dataset size, we train a Siamese Network to rank images in terms of image quality by using synthetically generated distortions for which relative image quality is known. These ranked image sets can be automatically generated without laborious human labeling. We then use fine-tuning to transfer the knowledge represented in the trained Siamese Network to a traditional CNN that estimates absolute image quality from single images. We demonstrate how our ap…
McGill University（麦吉尔大学）；École de technologie supérieure（高等技术学院）
In this letter, a very simple no-reference image quality assessment (NR-IQA) model for JPEG compressed images is proposed. The proposed metric called median of unique gradients (MUG) is based on the very simple facts of unique gradient magnitudes of JPEG compressed images. MUG is a parameterless metric and does not need training. Unlike other NR-IQAs, MUG is independent to block size and cropping. A more stable index called MUG+ is also introduced. The experimental results on six benchmark datasets of natural images and a benchmark dataset of synthetic images show that MUG is comparable to the…
Nanjing University of Information Science and Technology（南京信息工程大学）；Newcastle University（纽卡斯尔大学）；Inception Institute of Artificial Intelligence（起源人工智能研究院）
Age synthesis methods typically take a single image as input and use a specific number to control the age of the generated image. In this paper, we propose a novel framework taking two images as inputs, named dual-reference age synthesis (DRAS), which approaches the task differently; instead of using "hard" age information, i.e. a fixed number, our model determines the target age in a "soft" way, by employing a second reference image. Specifically, the proposed framework consists of an identity agent, an age agent and a generative adversarial network. It takes two images as input - an identity…
