Labeling semantic segmentation datasets is a costly and laborious process if compared with tasks like image classification and object detection. This is especially true for remote sensing applications that not only work with extremely high spatial resolution data but also commonly require the knowledge of experts of the area to perform the manual labeling. Data augmentation techniques help to improve deep learning models under the circumstance of few and imbalanced labeled samples. In this work, we propose a novel data augmentation method focused on exploring the spatial context of remote sensing semantic segmentation. This method, ChessMix, creates new synthetic images from the existing training set by mixing transformed mini-patches across the dataset in a chessboard-like grid. ChessMix prioritizes patches with more examples of the rarest classes to alleviate the imbalance problems. The results in three diverse well-known remote sensing datasets show that this is a promising approach that helps to improve the networks' performance, working especially well in datasets with few available data. The results also show that ChessMix is capable of improving the segmentation of objects with few labeled pixels when compared to the most common data augmentation methods widely used.
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）
Today deep convolutional neural networks (CNNs) push the limits for most computer vision problems, define trends, and set state-of-the-art results. In remote sensing tasks such as object detection and semantic segmentation, CNNs reach the SotA performance. However, for precise performance, CNNs require much high-quality training data. Rare objects and the variability of environmental conditions strongly affect prediction stability and accuracy. To overcome these data restrictions, it is common to consider various approaches including data augmentation techniques. This study focuses on the deve…
CP2M: Clustered-Patch-Mixed Mosaic Augmentation for Aerial Image Segmentation
Northwestern University（西北大学）；Carnegie Mellon University（卡内基梅隆大学）；The University of Hong Kong（香港大学）；Duke University（杜克大学）；University College Dublin（都柏林大学学院）；Beijing University of Technology（北京工业大学）；The ADAPT SFI Research Centre（ADAPT SFI研究中心）
TASAM: Terrain-and-Aware Segment Anything Model for Temporal-Scale Remote Sensing Segmentation
University of Alabama at Birmingham（阿拉巴马大学伯明翰分校）；Wuhan University（武汉大学）；Dalian University of Technology（大连理工大学）；University of Chinese Academy of Sciences（中国科学院大学）
Estimating Physical Information Consistency of Channel Data Augmentation for Remote Sensing Images
BIFOLD – Berlin Institute for the Foundations of Learning and Data（BIFOLD——柏林学习与数据基础研究所）；Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（柏林工业大学电气工程与计算机科学学院）
Adversarial Patch Attack for Ship Detection via Localized Augmentation
State Key Laboratory of Spatial Datum, College of Remote Sensing and Geoinformatics Engineering, Faculty of Geographical Science and Engineering, Henan University, Zhengzhou 450046, China（空间基准状态关键实验室，遥感与地理信息工程学院，地理科学与工程学院，河南大学，郑州450046，中国）；National Key Laboratory of Integrated Technology on Aircraft Control, Beihang University, Beijing 100191, China（航空控制集成技术国家实验室，北京航空航天大学，北京100191，中国）；School of Computer and Information Engineering, Henan University, Kaifeng 475004, China（计算机与信息工程学院，河南大学，开封475004，中国）；State Key Laboratory of Spatial Datum, Henan University, Zhengzhou 450046, China（空间基准状态关键实验室，河南大学，郑州450046，中国）
Diverse Instance Generation via Diffusion Models for Enhanced Few-Shot Object Detection in Remote Sensing Images
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Tsinghua University（清华大学）
Self-supervised methods have shown tremendous success in the field of computer vision, including applications in remote sensing and medical imaging. Most popular contrastive-loss based methods like SimCLR, MoCo, MoCo-v2 use multiple views of the same image by applying contrived augmentations on the image to create positive pairs and contrast them with negative examples. Although these techniques work well, most of these techniques have been tuned on ImageNet (and similar computer vision datasets). While there have been some attempts to capture a richer set of deformations in the positive sampl…
The difficulty in obtaining labeled data relevant to a given task is among the most common and well-known practical obstacles to applying deep learning techniques to new or even slightly modified domains. The data volumes required by the current generation of supervised learning algorithms typically far exceed what a human needs to learn and complete a given task. We investigate ways to expand a given labeled corpus of remote sensed imagery into a larger corpus using Generative Adversarial Networks (GANs). We then measure how these additional synthetic data affect supervised machine learning p…
Recent advances in supervised deep learning methods are enabling remote measurements of photoplethysmography-based physiological signals using facial videos. The performance of these supervised methods, however, are dependent on the availability of large labelled data. Contrastive learning as a self-supervised method has recently achieved state-of-the-art performances in learning representative data features by maximising mutual information between different augmented views. However, existing data augmentation techniques for contrastive learning are not designed to learn physiological signals …
Remote Sensing Imagery for Flood Detection: Exploration of Augmentation Strategies
Image Processing Department, Fraunhofer ITWM（弗劳恩霍夫ITWM图像处理部门）；Algorithms for Computer Vision, Imaging and Data Analysis Group, Hochschule Darmstadt（计算机视觉算法、成像与数据分析小组，达姆施塔特应用技术大学）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Camera-based remote photoplethysmography (rPPG) provides a non-contact way to measure physiological signals (e.g., heart rate) using facial videos. Recent deep learning architectures have improved the accuracy of such physiological measurement significantly, yet they are restricted by the diversity of the annotated videos. The existing datasets MMSE-HR, AFRL, and UBFC-RPPG contain roughly 10%, 0%, and 5% of dark-skinned subjects respectively. The unbalanced training sets result in a poor generalization capability to unseen subjects and lead to unwanted bias toward different demographic groups.…
Cloud and cloud shadow segmentation are fundamental processes in optical remote sensing image analysis. Current methods for cloud/shadow identification in geospatial imagery are not as accurate as they should, especially in the presence of snow and haze. This paper presents a deep learning-based framework for the detection of cloud/shadow in Landsat 8 images. Our method benefits from a convolutional neural network, Cloud-Net+ (a modification of our previously proposed Cloud-Net \cite{myigarss}) that is trained with a novel loss function (Filtered Jaccard Loss). The proposed loss function is mo…
Semi-supervised Semantic Segmentation for Remote Sensing Images via Multi-scale Uncertainty Consistency and Cross-Teacher-Student Attention
Faculty of Data Science, City University of Macau（数据科学学院，澳门城市大学）；WMG, University of Warwick（沃里克大学工程学院）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子、电气与通信工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
In this work, we revisit the weak-to-strong consistency framework, popularized by FixMatch from semi-supervised classification, where the prediction of a weakly perturbed image serves as supervision for its strongly perturbed version. Intriguingly, we observe that such a simple pipeline already achieves competitive results against recent advanced works, when transferred to our segmentation scenario. Its success heavily relies on the manual design of strong data augmentations, however, which may be limited and inadequate to explore a broader perturbation space. Motivated by this, we propose an …
Multisource Collaborative Domain Generalization for Cross-Scene Remote Sensing Image Classification
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；International Research Center of Big Data for Sustainable Development Goals（可持续发展大数据国际研究中心）；College of Resources and Environment, University of Chinese Academy of Sciences（中国科学院大学资源与环境学院）；School of Geographical Sciences, University of Bristol（布里斯托大学地理科学学院）；Beijing Institute of Remote Sensing Equipment（北京遥感设备研究所）；Hong Kong Baptist University（香港浸会大学）
Institute of Remote Sensing and Digital Earth, Chinese Academy of Sciences（中国科学院遥感与数字地球研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Lack of annotated samples greatly restrains the direct application of deep learning in remote sensing image scene classification. Although researches have been done to tackle this issue by data augmentation with various image transformation operations, they are still limited in quantity and diversity. Recently, the advent of the unsupervised learning based generative adversarial networks (GANs) bring us a new way to generate augmented samples. However, such GAN-generated samples are currently only served for training GANs model itself and for improving the performance of the discriminator in G…
Silesian University of Technology（西里西亚工业大学）；KP Labs（KP实验室）
Data augmentation is a popular technique which helps improve generalization capabilities of deep neural networks. It plays a pivotal role in remote-sensing scenarios in which the amount of high-quality ground truth data is limited, and acquiring new examples is costly or impossible. This is a common problem in hyperspectral imaging, where manual annotation of image data is difficult, expensive, and prone to human bias. In this letter, we propose online data augmentation of hyperspectral data which is executed during the inference rather than before the training of deep networks. This is in con…
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术学院）
The teleoperation of robotic systems in medical applications requires stable and convenient visual feedback for the operator. The most accessible approach to delivering visual information from the remote area is using cameras to transmit a video stream from the environment. However, such systems are sensitive to the camera resolution, limited viewpoints, and cluttered environment bringing additional mental demands to the human operator. The paper proposes a novel system of teleoperation based on an augmented virtual environment (VE). The region-based convolutional neural network (R-CNN) is app…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心（DLR）遥感技术研究所（IMF））；Technical University of Munich (TUM)（慕尼黑工业大学（TUM））
In deep learning research, self-supervised learning (SSL) has received great attention triggering interest within both the computer vision and remote sensing communities. While there has been a big success in computer vision, most of the potential of SSL in the domain of earth observation remains locked. In this paper, we provide an introduction to, and a review of the concepts and latest developments in SSL for computer vision in the context of remote sensing. Further, we provide a preliminary benchmark of modern SSL algorithms on popular remote sensing datasets, verifying the potential of SS…
University of Dongguk（东国大学）；Dongguk University Ilsan Hospital（东国大学一山医院）
Increasing numbers of patients with disabilities or elderly people with mobility issues often suffer from a pressure ulcer. The affected areas need regular checks, but they have a difficulty in accessing a hospital. Some remote diagnosis systems are being used for them, but there are limitations in checking a patient's status regularly. In this paper, we present a remote medical assistant that can help pressure ulcer management with image processing techniques. The proposed system includes a mobile application with a deep learning model for wound segmentation and analysis. As there are not eno…
Although convolutional neural networks have been proven to be an effective tool to generate high quality maps from remote sensing images, their performance significantly deteriorates when there exists a large domain shift between training and test data. To address this issue, we propose a new data augmentation approach that transfers the style of test data to training data using generative adversarial networks. Our semantic segmentation framework consists in first training a U-net from the real training data and then fine-tuning it on the test stylized fake training data generated by the propo…
Enhancing deep learning performance on burned area delineation from SPOT-6/7 imagery for emergency management
IRISA, Université Bretagne Sud, UMR 6074, Vannes, France（IRISA，布列塔尼大学，UMR 6074，瓦纳兹，法国）；University of Salzburg, Department of Geoinformatics, Salzburg, Austria（萨尔茨堡大学，地理信息学院，萨尔茨堡，奥地利）；ICube-SERTIT, Université de Strasbourg, Illkirch Graffenstaden, France（ICube-SERTIT，斯特拉斯堡大学，伊尔克里希格拉芬施塔登，法国）
Cleveland State University（克利夫兰州立大学）；Nanyang Technological University（南洋理工大学）；Wuhan University（武汉大学）；Beijing University of Technology（北京工业大学）；Brookhaven National Laboratory（布鲁克海文国家实验室）
The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. To the best of our knowledge, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but…
Center for Machine Vision and Signal Analysis, University of Oulu（奥卢大学机器视觉与信号分析中心）；Key Laboratory of Intelligent Information Processing, Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所智能信息处理重点实验室）；University of Chinese Academy of Sciences（中国科学院大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications (e.g., remote healthcare). Existing end-to-end rPPG and heart rate (HR) measurement methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and bad illumination). In this letter, we explore the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong end-to-end baseline (AutoHR) for remote HR measurement with neural architecture search (NAS). The proposed method inc…
A Deep-Learning Framework for Land-Sliding Classification from Remote Sensing Image
University of technology of Troyes（特鲁瓦技术大学）；ICC Dong Duong（东杜ong研究所）；Biosel AI（Biosel人工智能公司）；University of Technology Sydney（悉尼技术大学）；Austrian Institute of Technology（奥地利技术研究所）；Ho Chi Minh City University of Technology (HCMUT)（胡志明市技术大学）
University of California, Berkeley（加州大学伯克利分校）；University of Michigan, Ann Arbor（密歇根大学安娜堡分校）；Etegent Technologies, Ltd.（Etegent科技有限公司）；National Geospatial-Intelligence Agency（国家地理空间情报局）
