The increasing accessibility and precision of Earth observation satellite data offers considerable opportunities for industrial and state actors alike. This calls however for efficient methods able to process time-series on a global scale. Building on recent work employing multi-headed self-attention mechanisms to classify remote sensing time sequences, we propose a modification of the Temporal Attention Encoder. In our network, the channels of the temporal inputs are distributed among several compact attention heads operating in parallel. Each head extracts highly-specialized temporal features which are in turn concatenated into a single representation. Our approach outperforms other state-of-the-art time series classification algorithms on an open-access satellite image dataset, while using significantly fewer parameters and with a reduced computational complexity.
Characterizing the processes leading to deforestation is critical to the development and implementation of targeted forest conservation and management policies. In this work, we develop a deep learning model called ForestNet to classify the drivers of primary forest loss in Indonesia, a country with one of the highest deforestation rates in the world. Using satellite imagery, ForestNet identifies the direct drivers of deforestation in forest loss patches of any size. We curate a dataset of Landsat 8 satellite images of known forest loss events paired with driver annotations from expert interpr…
University of Virginia, School of Data Science（弗吉尼亚大学数据科学学院）
Hyperspectral Imagining is a type of digital imaging in which each pixel contains typically hundreds of wavelengths of light providing spectroscopic information about the materials present in the pixel. In this paper we provide classification methods for determining crop type in the USGS GHISACONUS data, which contains around 7,000 pixel spectra from the five major U.S. agricultural crops (winter wheat, rice, corn, soybeans, and cotton) collected by the NASA Hyperion satellite, and includes the spectrum, geolocation, crop type, and stage of growth for each pixel. We apply standard LDA and QDA …
Satellite imagery has long been an attractive data source that provides a wealth of information on human-inhabited areas. While super resolution satellite images are rapidly becoming available, little study has focused on how to extract meaningful information about human habitation patterns and economic scales from such data. We present READ, a new approach for obtaining essential spatial representation for any given district from high-resolution satellite imagery based on deep neural networks. Our method combines transfer learning and embedded statistics to efficiently learn critical spatial …
University of Missouri（密苏里大学）；Kennesaw State University（肯尼索州立大学）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Cybersickness is a common ailment associated with virtual reality (VR) user experiences. Several automated methods exist based on machine learning (ML) and deep learning (DL) to detect cybersickness. However, most of these cybersickness detection methods are perceived as computationally intensive and black-box methods. Thus, those techniques are neither trustworthy nor practical for deploying on standalone energy-constrained VR head-mounted devices (HMDs). In this work, we present an explainable artificial intelligence (XAI)-based framework, LiteVR, for cybersickness detection, explaining the …
Land use classification of low resolution spatial imagery is one of the most extensively researched fields in remote sensing. Despite significant advancements in satellite technology, high resolution imagery lacks global coverage and can be prohibitively expensive to procure for extended time periods. Accurately classifying land use change without high resolution imagery offers the potential to monitor vital aspects of global development agenda including climate smart agriculture, drought resistant crops, and sustainable land management. Utilizing a combination of capsule layers and long-short…
Hubei University of Technology（湖北工业大学）；Wuhan University（武汉大学）
Land cover classification is a multi-class segmentation task to classify each pixel into a certain natural or man-made category of the earth surface, such as water, soil, natural vegetation, crops, and human infrastructure. Limited by hardware computational resources and memory capacity, most existing studies preprocessed original remote sensing images by down sampling or cropping them into small patches less than 512*512 pixels before sending them to a deep neural network. However, down sampling images incurs spatial detail loss, renders small segments hard to discriminate, and reverses the s…
University of Naples Federico II（那不勒斯费德里科二世大学）；University of Maryland（马里兰大学）；University of Salzburg（萨尔茨堡大学）
The European Space Agency (ESA) defines an Earth Observation (EO) Level 2 product as a multispectral (MS) image corrected for geometric, atmospheric, adjacency and topographic effects, stacked with its scene classification map (SCM), whose legend includes quality layers such as cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To contribute toward filling an information gap from EO big data to the ESA EO Level 2 product, an original Stage 4 validation (Val) of the Satellite Image Automatic Mapper (SIAM) lightweight computer program …
University of Naples Federico II（那不勒斯费德里科二世大学）；University of Maryland（马里兰大学）；University of Salzburg（萨尔茨堡大学）
The European Space Agency (ESA) defines an Earth Observation (EO) Level 2 product as a multispectral (MS) image corrected for geometric, atmospheric, adjacency and topographic effects, stacked with its scene classification map (SCM) whose legend includes quality layers such as cloud and cloud-shadow. No ESA EO Level 2 product has ever been systematically generated at the ground segment. To contribute toward filling an information gap from EO big sensory data to the ESA EO Level 2 product, a Stage 4 validation (Val) of an off the shelf Satellite Image Automatic Mapper (SIAM) lightweight compute…
RSDehamba: Lightweight Vision Mamba for Remote Sensing Satellite Image Dehazing
College of Electronic Information Engineering, Shenyang Aerospace University（沈阳航空航天大学电子信息工程学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；School of Basic Sciences for Aviation, Naval Aviation University（海军航空大学航空基础学院）
Satellite imagery is important for many applications including disaster response, law enforcement, and environmental monitoring. These applications require the manual identification of objects and facilities in the imagery. Because the geographic expanses to be covered are great and the analysts available to conduct the searches are few, automation is required. Yet traditional object detection and classification algorithms are too inaccurate and unreliable to solve the problem. Deep learning is a family of machine learning algorithms that have shown promise for the automation of such tasks. It…
Timely recognition of plant pests from field images is significant to avoid potential losses of crop yields. Traditional convolutional neural network-based deep learning models demand high computational capability and require large labelled samples for each pest type for training. On the other hand, the existing lightweight network-based approaches suffer in correctly classifying the pests because of common characteristics and high similarity between multiple plant pests. In this work, a novel double attention-based lightweight deep learning architecture is proposed to automatically recognize …
Center for Computational and Data-Intensive Science and Engineering (CDISE)（计算与数据密集型科学与工程中心（CDISE））；Skolkovo Institute of Science and Technology (Skoltech)（斯科尔科沃科学技术学院（Skoltech））
Machine learning (ML) methods and neural networks (NN) are widely implemented for crop types recognition and classification based on satellite images. However, most of these studies use several multi-temporal images which could be inapplicable for cloudy regions. We present a comparison between the classical ML approaches and U-Net NN for classifying crops with a single satellite image. The results show the advantages of using field-wise classification over pixel-wise approach. We first used a Bayesian aggregation for field-wise classification and improved on 1.5% results between majority voti…
The increasingly common use of incidental satellite images for stereo reconstruction versus rigidly tasked binocular or trinocular coincident collection is helping to enable timely global-scale 3D mapping; however, reliable stereo correspondence from multi-date image pairs remains very challenging due to seasonal appearance differences and scene change. Promising recent work suggests that semantic scene segmentation can provide a robust regularizing prior for resolving ambiguities in stereo correspondence and reconstruction problems. To enable research for pairwise semantic stereo and multi-vi…
COVID-19 has led to hundreds of millions of cases and millions of deaths worldwide since its onset. The fight against this pandemic is on-going on multiple fronts. While vaccinations are picking up speed, there are still billions of unvaccinated people. In this fight against the virus, diagnosis of the disease and isolation of the patients to prevent any spread play a huge role. Machine Learning approaches have assisted in the diagnosis of COVID-19 cases by analyzing chest X-rays and CT-scan images of patients. To push algorithm development and research in this direction of radiological diagno…
Beyond Off-the-Shelf Models: A Lightweight and Accessible Machine Learning Pipeline for Ecologists Working with Image Data
Department of Statistics, LMU Munich（统计学系，慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Research Unit Wildlife Biology and Management, Bavarian State Institute for Forestry（林业野生动物生物学与管理研究组，巴伐利亚州林业研究所）
Lightweight, Pre-trained Transformers for Remote Sensing Timeseries
Mila – Quebec AI Institute（米拉-魁北克人工智能研究所）；McGill University（麦吉尔大学）；KU Leuven（鲁汶大学）；University of Maryland, College Park（马里兰大学帕克分校）；Arizona State University（亚利桑那州立大学）
Kyushu University（九州大学）；Indian Institute of Technology Delhi（德里印度理工学院）；Royal Holloway University of London（伦敦大学皇家霍洛威学院）
The Internet of Things (IoT) is an extension of the traditional Internet, which allows a very large number of smart devices, such as home appliances, network cameras, sensors and controllers to connect to one another to share information and improve user experiences. Current IoT devices are typically micro-computers for domain-specific computations rather than traditional functionspecific embedded devices. Therefore, many existing attacks, targeted at traditional computers connected to the Internet, may also be directed at IoT devices. For example, DDoS attacks have become very common in IoT e…
Universidade Tecnológica Federal do Paraná（巴拉那联邦理工大学）；University of South Florida（南佛罗里达大学）
The COVID-19 outbreak forced governments worldwide to impose lockdowns and quarantines to prevent virus transmission. As a consequence, there are disruptions in human and economic activities all over the globe. The recovery process is also expected to be rough. Economic activities impact social behaviors, which leave signatures in satellite images that can be automatically detected and classified. Satellite imagery can support the decision-making of analysts and policymakers by providing a different kind of visibility into the unfolding economic changes. In this work, we use a deep learning ap…
In the remote sensing community, Land Use Land Cover (LULC) classification with satellite imagery is a main focus of current research activities. Accurate and appropriate LULC classification, however, continues to be a challenging task. In this paper, we evaluate the performance of multi-temporal (monthly time series) compared to mono-temporal (single time step) satellite images for multi-label classification using supervised learning on the RapidAI4EO dataset. As a first step, we trained our CNN model on images at a single time step for multi-label classification, i.e. mono-temporal. We incor…
University of Oxford（牛津大学）；University of Cambridge（剑桥大学）；University of Edinburgh（爱丁堡大学）；Microsoft（微软）；University of Surrey（萨里大学）；University of Valencia（瓦伦西亚大学）；INRIA（法国国家信息与自动化研究所）；University College London（伦敦大学学院）
In this paper, we introduce RaVAEn, a lightweight, unsupervised approach for change detection in satellite data based on Variational Auto-Encoders (VAEs) with the specific purpose of on-board deployment. Applications such as disaster management enormously benefit from the rapid availability of satellite observations. Traditionally, data analysis is performed on the ground after all data is transferred - downlinked - to a ground station. Constraint on the downlink capabilities therefore affects any downstream application. In contrast, RaVAEn pre-processes the sampled data directly on the satell…
University of Sannio（萨尼奥大学）；Massachusetts Institute of Technology (MIT)（麻省理工学院（MIT））
In this paper, the authors aim to combine the latest state of the art models in image recognition with the best publicly available satellite images to create a system for landslide risk mitigation. We focus first on landslide detection and further propose a similar system to be used for prediction. Such models are valuable as they could easily be scaled up to provide data for hazard evaluation, as satellite imagery becomes increasingly available. The goal is to use satellite images and correlated data to enrich the public repository of data and guide disaster relief efforts for locating precis…
University of South Florida（南佛罗里达大学）；The Federal University of Technology - Paraná（巴拉那联邦理工大学）
We describe a strategy for detection and classification of man-made objects in large high-resolution satellite photos under computational resource constraints. We detect and classify candidate objects by using five pipelines of convolutional neural network processing (CNN), run in parallel. Each pipeline has its own unique strategy for fine tunning parameters, proposal region filtering, and dealing with image scales. The conflicting region proposals are merged based on region confidence and not just based on overlap areas, which improves the quality of the final bounding-box regions selected. …
FLNet: Flood-Induced Agriculture Damage Assessment using Super Resolution of Satellite Images
Annam.AI CoE, MoE, Indian Institute of Technology Ropar, Punjab, India（Annam.AI联合研究所、教育部、印度理工学院罗帕尔分校）；Department of Mathematical Sciences, Rajiv Gandhi Institute of Petroleum Technology, Jais, UP, India（数学科学系、拉贾吉安石油技术学院）；Department of Computer Science and Engineering, Indian Institute of Technology Ropar, Punjab, India（计算机科学与工程系、印度理工学院罗帕尔分校）
Climate change has been a common interest and the forefront of crucial political discussion and decision-making for many years. Shallow clouds play a significant role in understanding the Earth's climate, but they are challenging to interpret and represent in a climate model. By classifying these cloud structures, there is a better possibility of understanding the physical structures of the clouds, which would improve the climate model generation, resulting in a better prediction of climate change or forecasting weather update. Clouds organise in many forms, which makes it challenging to build…
Fast model inference and training on-board of Satellites
University of Oxford（牛津大学）；Trillium Technologies（特里利姆技术公司）；University of Valencia（瓦伦西亚大学）；University of Surrey（萨里大学）；University of New South Wales（新南威尔士大学）；D-Orbit（D-Orbit公司）；European Space Agency（欧洲空间局）
本文在 ION SCV004 卫星上部署轻量级 VAE 模型 RaVAEn，比较 CPU 与 Myriad VPU，实现快速在轨编码、少样本训练及多任务推理。
