In machine learning, the term active learning regroups techniques that aim at selecting the most useful data to label from a large pool of unlabelled examples. While supervised deep learning techniques have shown to be increasingly efficient on many applications, they require a huge number of labelled examples to reach operational performances. Therefore, the labelling effort linked to the creation of the datasets required is also increasing. When working on defense-related remote sensing applications, labelling can be challenging due to the large areas covered and often requires military experts who are rare and whose time is primarily dedicated to operational needs. Limiting the labelling effort is thus of utmost importance. This study aims at reviewing the most relevant active learning techniques to be used for object detection on very high resolution imagery and shows an example of the value of such techniques on a relevant operational use case: aircraft detection.
We introduce in this paper a novel active learning algorithm for satellite image change detection. The proposed solution is interactive and based on a question and answer model, which asks an oracle (annotator) the most informative questions about the relevance of sampled satellite image pairs, and according to the oracle's responses, updates a decision function iteratively. We investigate a novel framework which models the probability that samples are relevant; this probability is obtained by minimizing an objective function capturing representativity, diversity and ambiguity. Only data with …
Road network and building footprint extraction is essential for many applications such as updating maps, traffic regulations, city planning, ride-hailing, disaster response \textit{etc}. Mapping road networks is currently both expensive and labor-intensive. Recently, improvements in image segmentation through the application of deep neural networks has shown promising results in extracting road segments from large scale, high resolution satellite imagery. However, significant challenges remain due to lack of enough labeled training data needed to build models for industry grade applications. I…
Remote sensing data is crucial for applications ranging from monitoring forest fires and deforestation to tracking urbanization. Most of these tasks require dense pixel-level annotations for the model to parse visual information from limited labeled data available for these satellite images. Due to the dearth of high-quality labeled training data in this domain, there is a need to focus on semi-supervised techniques. These techniques generate pseudo-labels from a small set of labeled examples which are used to augment the labeled training set. This makes it necessary to have a highly represent…
Orion Lab, National Observatory of Athens & National Technical University of Athens（Orion实验室、雅典国家天文观测站及雅典技术大学）；Department of Informatics and Telematics, Harokopio University of Athens（信息与电信系、哈罗基奥大学）
In this paper, we present a ship detection pipeline for low-cost medium resolution satellite optical imagery obtained from ESA Sentinel-2 and Planet Labs Dove constellations. This optical satellite imagery is readily available for any place on Earth and underutilized in the maritime domain, compared to existing solutions based on synthetic-aperture radar (SAR) imagery. We developed a ship detection method based on a state-of-the-art deep-learning-based object detection method which was developed and evaluated on a large-scale dataset that was collected and automatically annotated with the help…
We explore the application of super-resolution techniques to satellite imagery, and the effects of these techniques on object detection algorithm performance. Specifically, we enhance satellite imagery beyond its native resolution, and test if we can identify various types of vehicles, planes, and boats with greater accuracy than native resolution. Using the Very Deep Super-Resolution (VDSR) framework and a custom Random Forest Super-Resolution (RFSR) framework we generate enhancement levels of 2x, 4x, and 8x over five distinct resolutions ranging from 30 cm to 4.8 meters. Using both native an…
Gulbarga University（古尔巴加大学）；S. V. College, Udgir（乌迪尔S.V.学院）
In this paper, we study efficient and reliable automatic extraction algorithm to find out the open space area from the high resolution urban satellite imagery, and to detect changes from the extracted open space area during the period 2003, 2006 and 2008. This automatic extraction and change detection algorithm uses some filters, segmentation and grouping that are applied on satellite images. The resultant images may be used to calculate the total available open space area and the built up area. It may also be used to compare the difference between present and past open space area using histor…
Ensembling is a method that aims to maximize the detection performance by fusing individual detectors. While rarely mentioned in deep-learning articles applied to remote sensing, ensembling methods have been widely used to achieve high scores in recent data science com-petitions, such as Kaggle. The few remote sensing articles mentioning ensembling mainly focus on mid resolution images and earth observation applications such as land use classification, but never on Very High Resolution (VHR) images for defense-related applications or object detection.This study aims at reviewing the most relev…
Access to high resolution satellite imagery has dramatically increased in recent years as several new constellations have entered service. High revisit frequencies as well as improved resolution has widened the use cases of satellite imagery to areas such as humanitarian relief and even Search and Rescue (SaR). We propose a novel remote sensing object detection dataset for deep learning assisted SaR. This dataset contains only small objects that have been identified as potential targets as part of a live SaR response. We evaluate the application of popular object detection models to this datas…
In this paper, we devise a novel interactive satellite image change detection algorithm based on active learning. The proposed framework is iterative and relies on a question and answer model which asks the oracle (user) questions about the most informative display (subset of critical images), and according to the user's responses, updates change detections. The contribution of our framework resides in a novel display model which selects the most representative and diverse virtual exemplars that adversely challenge the learned change detection functions, thereby leading to highly discriminatin…
This paper presents an aligned multi-temporal and multi-resolution satellite image dataset for research in change detection. We expect our dataset to be useful to researchers who want to fuse information from multiple satellites for detecting changes on the surface of the earth that may not be fully visible in any single satellite. The dataset we present was created by augmenting the SpaceNet-7 dataset with temporally parallel stacks of Landsat and Sentinel images. The SpaceNet-7 dataset consists of time-sequenced Planet images recorded over 101 AOIs (Areas-of-Interest). In our dataset, for ea…
Greenhouse segmentation has pivotal importance for climate-smart agricultural land-use planning. Deep learning-based approaches provide state-of-the-art performance in natural image segmentation. However, semantic segmentation on high-resolution optical satellite imagery is a challenging task because of the complex environment. In this paper, a sound methodology is proposed for pixel-wise classification on images acquired by the Azersky (SPOT-7) optical satellite. In particular, customized variations of U-Net-like architectures are employed to identify greenhouses. Two models are proposed whic…
Identifying the locations and footprints of buildings is vital for many practical and scientific purposes. Such information can be particularly useful in developing regions where alternative data sources may be scarce. In this work, we describe a model training pipeline for detecting buildings across the entire continent of Africa, using 50 cm satellite imagery. Starting with the U-Net model, widely used in satellite image analysis, we study variations in architecture, loss functions, regularization, pre-training, self-training and post-processing that increase instance segmentation performanc…
In this paper, we introduce a novel interactive satellite image change detection algorithm based on active learning. The proposed approach is iterative and asks the user (oracle) questions about the targeted changes and according to the oracle's responses updates change detections. We consider a probabilistic framework which assigns to each unlabeled sample a relevance measure modeling how critical is that sample when training change detection functions. These relevance measures are obtained by minimizing an objective function mixing diversity, representativity and uncertainty. These criteria …
School of Automation, Beijing Institute of Technology（北京理工大学自动化学院）
Oriented object detection in remote sensing images has made great progress in recent years. However, most of the current methods only focus on detecting targets, and cannot distinguish fine-grained objects well in complex scenes. In this technical report, we analyzed the key issues of fine-grained object recognition, and use an oriented feature alignment network (OFA-Net) to achieve high-performance fine-grained oriented object recognition in optical remote sensing images. OFA-Net achieves accurate object localization through a rotated bounding boxes refinement module. On this basis, the bound…
Detection of small objects in large swaths of imagery is one of the primary problems in satellite imagery analytics. While object detection in ground-based imagery has benefited from research into new deep learning approaches, transitioning such technology to overhead imagery is nontrivial. Among the challenges is the sheer number of pixels and geographic extent per image: a single DigitalGlobe satellite image encompasses >64 km2 and over 250 million pixels. Another challenge is that objects of interest are minuscule (often only ~10 pixels in extent), which complicates traditional computer vis…
Satellite image change detection aims at finding occurrences of targeted changes in a given scene taken at different instants. This task is highly challenging due to the acquisition conditions and also to the subjectivity of changes. In this paper, we investigate satellite image change detection using active learning. Our method is interactive and relies on a question and answer model which asks the oracle (user) questions about the most informative display (dubbed as virtual exemplars), and according to the user's responses, updates change detections. The main contribution of our method consi…
Indraprastha Institute of Information Technology, Delhi（德里英迪拉普拉斯塔信息技术学院）；M S Swaminathan Research Foundation（M·S·斯瓦米纳坦研究基金会）
The integration of the modern Machine Learning (ML) models into remote sensing and agriculture has expanded the scope of the application of satellite images in the agriculture domain. In this paper, we present how the accuracy of crop type identification improves as we move from medium-spatiotemporal-resolution (MSTR) to high-spatiotemporal-resolution (HSTR) satellite images. We further demonstrate that high spectral resolution in satellite imagery can improve prediction performance for low spatial and temporal resolutions (LSTR) images. The F1-score is increased by 7% when using multispectral…
Urban material recognition in remote sensing imagery is a highly relevant, yet extremely challenging problem due to the difficulty of obtaining human annotations, especially on low resolution satellite images. To this end, we propose an unsupervised domain adaptation based approach using adversarial learning. We aim to harvest information from smaller quantities of high resolution data (source domain) and utilize the same to super-resolve low resolution imagery (target domain). This can potentially aid in semantic as well as material label transfer from a richly annotated source to a target do…
Semantic segmentation of satellite imagery is a common approach to identify patterns and detect changes around the planet. Most of the state-of-the-art semantic segmentation models are trained in a fully supervised way using Convolutional Neural Network (CNN). The generalization property of CNN is poor for satellite imagery because the data can be very diverse in terms of landscape types, image resolutions, and scarcity of labels for different geographies and seasons. Hence, the performance of CNN doesn't translate well to images from unseen regions or seasons. Inspired by Conditional Generati…
The earth observation industry provides satellite imagery with high spatial resolution and short revisit time. To allow efficient operational employment of these images, automating certain tasks has become necessary. In the defense domain, aircraft detection on satellite imagery is a valuable tool for analysts. Obtaining high performance detectors on such a task can only be achieved by leveraging deep learning and thus us-ing a large amount of labeled data. To obtain labels of a high enough quality, the knowledge of military experts is needed.We propose a hybrid clustering active learning meth…
Image resolution is an important criterion for many applications based on satellite imagery. In this work, we adapt a state-of-the-art kernel regression technique for smartphone camera burst super-resolution to satellites. This technique leverages the local structure of the image to optimally steer the fusion kernels, limiting blur in the final high-resolution prediction, denoising the image, and recovering details up to a zoom factor of 2. We extend this approach to the multi-exposure case to predict from a sequence of multi-exposure low-resolution frames a high-resolution and noise-free one.…
Land use classification of low resolution spatial imagery is one of the most extensively researched fields in remote sensing. Despite significant advancements in satellite technology, high resolution imagery lacks global coverage and can be prohibitively expensive to procure for extended time periods. Accurately classifying land use change without high resolution imagery offers the potential to monitor vital aspects of global development agenda including climate smart agriculture, drought resistant crops, and sustainable land management. Utilizing a combination of capsule layers and long-short…
Bangladesh University of Engineering and Technology（孟加拉国工程技术大学）
In the modern world, satellite images play a key role in forest management and degradation monitoring. For a precise quantification of forest land cover changes, the availability of spatially fine resolution data is a necessity. Since 1972, NASAs LANDSAT Satellites are providing terrestrial images covering every corner of the earth, which have been proved to be a highly useful resource for terrestrial change analysis and have been used in numerous other sectors. However, freely accessible satellite images are, generally, of medium to low resolution which is a major hindrance to the precision o…
In machine learning based single image super-resolution, the degradation model is embedded in training data generation. However, most existing satellite image super-resolution methods use a simple down-sampling model with a fixed kernel to create training images. These methods work fine on synthetic data, but do not perform well on real satellite images. We propose a realistic training data generation model for commercial satellite imagery products, which includes not only the imaging process on satellites but also the post-process on the ground. We also propose a convolutional neural network …
Future Processing（未来处理公司）；Silesian University of Technology（西里西亚工业大学）
The capabilities of super-resolution reconstruction (SRR)---techniques for enhancing image spatial resolution---have been recently improved significantly by the use of deep convolutional neural networks. Commonly, such networks are learned using huge training sets composed of original images alongside their low-resolution counterparts, obtained with bicubic downsampling. In this paper, we investigate how the SRR performance is influenced by the way such low-resolution training data are obtained, which has not been explored up to date. Our extensive experimental study indicates that the trainin…
Satellite images often contain a significant level of sensitive data compared to ground-view images. That is why satellite images are more likely to be intentionally manipulated to hide specific objects and structures. GAN-based approaches have been employed to create forged images with two major problems: (i) adding a new object to the scene to hide a specific object or region may create unrealistic merging with surrounding areas; and (ii) using masks on color feature images has proven to be unsuccessful in GAN-based object removal. In this paper, we tackle the problem of object removal in hi…
In the 21st century, Aerial and satellite images are information rich. They are also complex to analyze. For GIS systems, many features require fast and reliable extraction of open space area from high resolution satellite imagery. In this paper we will study efficient and reliable automatic extraction algorithm to find out the open space area from the high resolution urban satellite imagery. This automatic extraction algorithm uses some filters and segmentations and grouping is applying on satellite images. And the result images may use to calculate the total available open space area and the…
Indian Institute of Technology, Roorkee（印度理工学院鲁尔基分校）
The presented study is an eye tracking experiment for high-resolution satellite (HRS) images. The reported experiment explores the Area Of Interest (AOI) based analysis of eye fixation data for complex HRS images. The study reflects the requisite of reference data for bottom-up saliency-based segmentation and the struggle of eye tracking data analysis for complex satellite images. The intended fixation data analysis aims towards the reference data creation for bottom-up saliency-based segmentation of high-resolution satellite images. The analytical outcome of this experimental study provides a…
Element AI；Mila；Université de Montréal（蒙特利尔大学）；McGill University（麦吉尔大学）
Generative deep learning has sparked a new wave of Super-Resolution (SR) algorithms that enhance single images with impressive aesthetic results, albeit with imaginary details. Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views. This is important for satellite monitoring of human impact on the planet -- from deforestation, to human rights violations -- that depend on reliable imagery. To this end, we present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end…
The major drawbacks with Satellite Images are low resolution, Low resolution makes it difficult to identify the objects present in Satellite images. We have experimented with several deep models available for Single Image Superresolution on the SpaceNet dataset and have evaluated the performance of each of them on the satellite image data. We will dive into the recent evolution of the deep models in the context of SISR over the past few years and will present a comparative study between these models. The entire Satellite image of an area is divided into equal-sized patches. Each patch will be …
Super-resolution aims at increasing image resolution by algorithmic means and has progressed over the recent years due to advances in the fields of computer vision and deep learning. Convolutional Neural Networks based on a variety of architectures have been applied to the problem, e.g. autoencoders and residual networks. While most research focuses on the processing of photographs consisting only of RGB color channels, little work can be found concentrating on multi-band, analytic satellite imagery. Satellite images often include a panchromatic band, which has higher spatial resolution but lo…
Given enough multi-view image corresponding points (also called tie points) and ground control points (GCP), bundle adjustment for high-resolution satellite images is used to refine the orientations or most often used geometric parameters Rational Polynomial Coefficients (RPC) of each satellite image in a unified geodetic framework, which is very critical in many photogrammetry and computer vision applications. However, the growing number of high resolution spaceborne optical sensors has brought two challenges to the bundle adjustment: 1) images come from different satellite cameras may have d…
National Institute of Research and Training in Telecommunications (INICTEL-UNI)（国家电信研究与培训学院（INICTEL-UNI））；National University of Engineering（国立工程大学）
Segmenting clouds in high-resolution satellite images is an arduous and challenging task due to the many types of geographies and clouds a satellite can capture. Therefore, it needs to be automated and optimized, specially for those who regularly process great amounts of satellite images, such as governmental institutions. In that sense, the contribution of this work is twofold: We present the CloudPeru2 dataset, consisting of 22,400 images of 512x512 pixels and their respective hand-drawn cloud masks, as well as the proposal of an end-to-end segmentation method for clouds using a Convolutiona…
Massive amounts of satellite data have been gathered over time, holding the potential to unveil a spatiotemporal chronicle of the surface of Earth. These data allow scientists to investigate various important issues, such as land use changes, on a global scale. However, not all land-use phenomena are equally visible on satellite imagery. In particular, the creation of an inventory of the planet's road infrastructure remains a challenge, despite being crucial to analyze urbanization patterns and their impact. Towards this end, this work advances data-driven approaches for the automatic identifi…
As Super-Resolution (SR) has matured as a research topic, it has been applied to additional topics beyond image reconstruction. In particular, combining classification or object detection tasks with a super-resolution preprocessing stage has yielded improvements in accuracy especially with objects that are small relative to the scene. While SR has shown promise, a study comparing SR and naive upscaling methods such as Nearest Neighbors (NN) interpolation when applied as a preprocessing step for object detection has not been performed. We apply the topic to satellite data and compare the Multi-…
National University of Water and Environmental Engineering（国立水利与环境工程大学）；EOS Data Analytics Ukraine（乌克兰EOS数据分析公司）；Oles Honchar Dnipro National University（奥列西·贡恰尔第聂伯国立大学）
In this paper we discuss a new variational approach to the Date Fusion problem of multi-spectral satellite images from Sentinel-2 and MODIS that have been captured at different resolution level and, arguably, on different days. The crucial point of our approach that the MODIS image is cloud-free whereas the images from Sentinel-2 can be corrupted by clouds or noise.
The Ohio State University（俄亥俄州立大学）；Byrd Polar and Climate Research Center（伯德极地与气候研究中心）
Remote sensing of the Earth's surface water is critical in a wide range of environmental studies, from evaluating the societal impacts of seasonal droughts and floods to the large-scale implications of climate change. Consequently, a large literature exists on the classification of water from satellite imagery. Yet, previous methods have been limited by 1) the spatial resolution of public satellite imagery, 2) classification schemes that operate at the pixel level, and 3) the need for multiple spectral bands. We advance the state-of-the-art by 1) using commercial imagery with panchromatic and …
