We tackle the problem of image inpainting in the remote sensing domain. Remote sensing images possess high resolution and geographical variations, that render the conventional inpainting methods less effective. This further entails the requirement of models with high complexity to sufficiently capture the spectral, spatial and textural nuances within an image, emerging from its high spatial variability. To this end, we propose a novel inpainting method that individually focuses on each aspect of an image such as edges, colour and texture using a task specific GAN. Moreover, each individual GAN also incorporates the attention mechanism that explicitly extracts the spectral and spatial features. To ensure consistent gradient flow, the model uses residual learning paradigm, thus simultaneously working with high and low level features. We evaluate our model, alongwith previous state of the art models, on the two well known remote sensing datasets, Open Cities AI and Earth on Canvas, and achieve competitive performance.
The difficulty in obtaining labeled data relevant to a given task is among the most common and well-known practical obstacles to applying deep learning techniques to new or even slightly modified domains. The data volumes required by the current generation of supervised learning algorithms typically far exceed what a human needs to learn and complete a given task. We investigate ways to expand a given labeled corpus of remote sensed imagery into a larger corpus using Generative Adversarial Networks (GANs). We then measure how these additional synthetic data affect supervised machine learning p…
Despite recent advances in deep-learning based semantic segmentation, automatic building detection from remotely sensed imagery is still a challenging problem owing to large variability in the appearance of buildings across the globe. The errors occur mostly around the boundaries of the building footprints, in shadow areas, and when detecting buildings whose exterior surfaces have reflectivity properties that are very similar to those of the surrounding regions. To overcome these problems, we propose a generative adversarial network based segmentation framework with uncertainty attention unit …
UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotations
In the U.S., corn is the most produced crop and has been an essential part of the American diet. To meet the demand for supply chain management and regional food security, accurate and timely large-scale corn yield prediction is attracting more attention in precision agriculture. Recently, remote sensing technology and machine learning methods have been widely explored for crop yield prediction. Currently, most county-level yield prediction models use county-level mean variables for prediction, ignoring much detailed information. Moreover, inconsistent spatial resolution between crop area and …
Estimating the heightmaps of buildings and vegetation in single remotely sensed images is a challenging problem. Effective solutions to this problem can comprise the stepping stone for solving complex and demanding problems that require 3D information of aerial imagery in the remote sensing discipline, which might be expensive or not feasible to require. We propose a task-focused Deep Learning (DL) model that takes advantage of the shadow map of a remotely sensed image to calculate its heightmap. The shadow is computed efficiently and does not add significant computation complexity. The model …
Lancaster University（兰卡斯特大学）；UK Centre for Ecology & Hydrology（英国生态与水文中心）；Wuhan University（武汉大学）；University of Southampton（南安普顿大学）
Semantic segmentation of remotely sensed images plays an important role in land resource management, yield estimation, and economic assessment. U-Net, a deep encoder-decoder architecture, has been used frequently for image segmentation with high accuracy. In this Letter, we incorporate multi-scale features generated by different layers of U-Net and design a multi-scale skip connected and asymmetric-convolution-based U-Net (MACU-Net), for segmentation using fine-resolution remotely sensed images. Our design has the following advantages: (1) The multi-scale skip connections combine and realign s…
National Technical University of Athens（雅典国家技术大学）；National Centre for Scientific Research Demokritos（德谟克利特国家科学研究中心）
Procedural 3D Terrain generation has become a necessity in open world games, as it can provide unlimited content, through a functionally infinite number of different areas, for players to explore. In our approach, we use Generative Adversarial Networks (GAN) to yield realistic 3D environments based on the distribution of remotely sensed images of landscapes, captured by satellites or drones. Our task consists of synthesizing a random but plausible RGB satellite image and generating a corresponding Height Map in the form of a 3D point cloud that will serve as an appropriate mesh of the landscap…
This paper presents an improved edge detection algorithm for facial and remotely sensed images using vector order statistics. The developed algorithm processes colored images directly without been converted to gray scale. A number of the existing algorithms converts the colored images into gray scale before detection of edges. But this process leads to inaccurate precision of recognized edges, thus producing false and broken edges in the output edge map. Facial and remotely sensed images consist of curved edge lines which have to be detected continuously to prevent broken edges. In order to de…
Key Laboratory of Technology in Geo-spatial Information Processing and Application System, Chinese Academy of Sciences（中国科学院空间信息处理与应用系统技术重点实验室）
With the development of deep learning, supervised learning has frequently been adopted to classify remotely sensed images using convolutional networks (CNNs). However, due to the limited amount of labeled data available, supervised learning is often difficult to carry out. Therefore, we proposed an unsupervised model called multiple-layer feature-matching generative adversarial networks (MARTA GANs) to learn a representation using only unlabeled data. MARTA GANs consists of both a generative model $G$ and a discriminative model $D$. We treat $D$ as a feature extractor. To fit the complex prope…
Institute of Informatics and Telecommunications, NCSR Demokritos（希腊德谟克利特国家研究中心信息与通信研究所）；University of West Attica（西阿提卡大学）；Helenic Air Force Academy（希腊空军学院）；The City College of New York（纽约市立学院）；Map Ltd（Map有限公司）
In this work, very deep super-resolution (VDSR) method is presented for improving the spatial resolution of remotely sensed (RS) images for scale factor 4. The VDSR net is re-trained with Sentinel-2 images and with drone aero orthophoto images, thus becomes RS-VDSR and Aero-VDSR, respectively. A novel loss function, the Var-norm estimator, is proposed in the regression layer of the convolutional neural network during re-training and prediction. According to numerical and optical comparisons, the proposed nets RS-VDSR and Aero-VDSR can outperform VDSR during prediction with RS images. RS-VDSR o…
ABES Engineering College（ABES工程学院）；Jaypee Institute of Information Technology（贾皮信息技术学院）
With the advent of remote sensing satellites, a huge repository of remotely sensed images is available. Change detection in remotely sensed images has been an active research area as it helps us understand the transitions that are taking place on the Earths surface. This paper discusses the methods and their classifications proposed by various researchers for change detection. Since use of soft computing based techniques are now very popular among research community, this paper also presents a classification based on learning techniques used in soft-computing methods for change detection.
Despite the plethora of successful Super-Resolution Reconstruction (SRR) models applied to natural images, their application to remote sensing imagery tends to produce poor results. Remote sensing imagery is often more complicated than natural images and has its peculiarities such as being of lower resolution, it contains noise, and often depicting large textured surfaces. As a result, applying non-specialized SRR models on remote sensing imagery results in artifacts and poor reconstructions. To address these problems, this paper proposes an architecture inspired by previous research work, int…
Stanford University（斯坦福大学）；Tata Research Development and Design Centre（塔塔研究开发与设计中心）
Urban metabolism is an active field of research that deals with the estimation of emissions and resource consumption from urban regions. The analysis could be carried out through a manual surveyor by the implementation of elegant machine learning algorithms. In this exploratory work, we estimate the water consumption by the buildings in the region captured by satellite imagery. To this end, we break our analysis into three parts: i) Identification of building pixels, given a satellite image, followed by ii) identification of the building type (residential/non-residential) from the building pix…
University of Maryland（马里兰大学）；Johns Hopkins University（约翰斯·霍普金斯大学）；US Census Bureau（美国人口普查局）
We develop an algorithm for single-image superresolution of remotely sensed data, based on the discrete shearlet transform. The shearlet transform extracts directional features of signals, and is known to provide near-optimally sparse representations for a broad class of images. This often leads to superior performance in edge detection and image representation when compared to isotropic frames. We justify the use of shearlets mathematically, before presenting a denoising single-image superresolution algorithm that combines the shearlet transform with sparse mixing estimators (SME). Our algori…
Resampling techniques are being widely used at different stages of satellite image processing. The existing methodologies cannot perfectly recover features from a completely under sampled image and hence an intelligent adaptive resampling methodology is required. We address these issues and adopt an error metric from the available literature to define interpolation quality. We also propose a new resampling scheme that adapts itself with regard to the pixel and texture variation in the image. The proposed CNN based hybrid method has been found to perform better than the existing methods as it a…
Quantitative analysis of channel networks plays an important role in river studies. To provide a quantitative representation of channel networks, we propose a new method that extracts channels from remotely sensed images and estimates their widths. Our fully automated method is based on a recently proposed Multiscale Singularity Index that responds strongly to curvilinear structures but weakly to edges. The algorithm produces a channel map, using a single image where water and non-water pixels have contrast, such as a Landsat near-infrared band image or a water index defined on multiple bands.…
Politecnico di Torino（都灵理工大学）；PIC4SeR（都灵理工大学跨部门服务机器人中心）
Convolutional Neural Networks (CNNs) have been consistently proved state-of-the-art results in image Super-Resolution (SR), representing an exceptional opportunity for the remote sensing field to extract further information and knowledge from captured data. However, most of the works published in the literature have been focusing on the Single-Image Super-Resolution problem so far. At present, satellite based remote sensing platforms offer huge data availability with high temporal resolution and low spatial resolution. In this context, the presented research proposes a novel residual attention…
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology and Hydrology（英国生态与水文中心）
Semantic segmentation using fine-resolution remotely sensed images plays a critical role in many practical applications, such as urban planning, environmental protection, natural and anthropogenic landscape monitoring, etc. However, the automation of semantic segmentation, i.e., automatic categorization/labeling and segmentation is still a challenging task, particularly for fine-resolution images with huge spatial and spectral complexity. Addressing such a problem represents an exciting research field, which paves the way for scene-level landscape pattern analysis and decision making. In this …
Remote sensing techniques are widely used for land cover classification and urban analysis. The availability of high resolution remote sensing imagery limits the level of classification accuracy attainable from pixel-based approach. In this paper object-based classification scheme based on a hierarchical support vector machine is introduced. By combining spatial and spectral information, the amount of overlap between classes can be decreased; thereby yielding higher classification accuracy and more accurate land cover maps. We have adopted certain automatic approaches based on the advanced tec…
Remote sensing research focusing on feature selection has long attracted the attention of the remote sensing community because feature selection is a prerequisite for image processing and various applications. Different feature selection methods have been proposed to improve the classification accuracy. They vary from basic search techniques to clonal selections, and various optimal criteria have been investigated. Recently, methods using dependence-based measures have attracted much attention due to their ability to deal with very high dimensional datasets. However, these methods are based on…
Band selection is a great challenging task in the classification of hyperspectral remotely sensed images HSI. This is resulting from its high spectral resolution, the many class outputs and the limited number of training samples. For this purpose, this paper introduces a new filter approach for dimension reduction and classification of hyperspectral images using information theoretic (normalized mutual information) and support vector machines SVM. This method consists to select a minimal subset of the most informative and relevant bands from the input datasets for better classification efficie…
Geo-informatics and Space Technology Development Agency (GISTDA)（地理信息与空间技术发展局）
Traditional image classification techniques often produce unsatisfactory results when applied to high spatial resolution data because classes in high resolution images are not spectrally homogeneous. Texture offers an alternative source of information for classifying these images. This paper evaluates a recently developed, computationally simple texture metric called Weber Local Descriptor (WLD) for use in classifying high resolution QuickBird panchromatic data. We compared WLD with state-of-the art texture descriptors (TD) including Local Binary Pattern (LBP) and its rotation-invariant versio…
Sharif University of Technology（谢里夫理工大学）；University of Tehran（德黑兰大学）
In recent years, unmanned aircraft systems (UASs) are frequently used in many different applications of photogrammetry such as building damage monitoring, archaeological mapping and vegetation monitoring. In this paper, a new state-of-the-art vertical take-off and landing fixed-wing UAS is proposed to robust photogrammetry missions, called SAMA-VTOL. In this study, the capability of SAMA-VTOL is investigated for generating orthophoto. The major stages are including designing, building and experimental scenario. First, a brief description of design and build is introduced. Next, an experiment w…
A generalised pre-training strategy for deep learning networks in semantic segmentation of remotely sensed images
Department of Civil Engineering, Xi’an Jiaotong-Liverpool University（西安交通大学利物浦大学土木工程系）；CSIRO Mineral Resources（澳大利亚联邦科学与工业研究组织矿物资源部门）；Department of Infrastructure Engineering, The University of Melbourne（墨尔本大学基础设施工程系）
Advanced machine learning techniques have been used in remote sensing (RS) applications such as crop mapping and yield prediction, but remain under-utilized for tracking crop progress. In this study, we demonstrate the use of agronomic knowledge of crop growth drivers in a Long Short-Term Memory-based, domain-guided neural network (DgNN) for in-season crop progress estimation. The DgNN uses a branched structure and attention to separate independent crop growth drivers and capture their varying importance throughout the growing season. The DgNN is implemented for corn, using RS data in Iowa for…
The Ohio State University（俄亥俄州立大学）；Michigan Technological University（密歇根理工大学）
As an important application in remote sensing, landcover classification remains one of the most challenging tasks in very-high-resolution (VHR) image analysis. As the rapidly increasing number of Deep Learning (DL) based landcover methods and training strategies are claimed to be the state-of-the-art, the already fragmented technical landscape of landcover mapping methods has been further complicated. Although there exists a plethora of literature review work attempting to guide researchers in making an informed choice of landcover mapping methods, the articles either focus on the review of ap…
At least a quarter of the warming that the Earth is experiencing today is due to anthropogenic methane emissions. There are multiple satellites in orbit and planned for launch in the next few years which can detect and quantify these emissions; however, to attribute methane emissions to their sources on the ground, a comprehensive database of the locations and characteristics of emission sources worldwide is essential. In this work, we develop deep learning algorithms that leverage freely available high-resolution aerial imagery to automatically detect oil and gas infrastructure, one of the la…
New York University（纽约大学）；University of the Philippines Diliman（菲律宾大学迪里曼分校）；University of Washington（华盛顿大学）；German Corporation for International Cooperation GmbH (GIZ)（德国国际合作机构（GIZ））
Rapid damage assessment is of crucial importance to emergency responders during hurricane events, however, the evaluation process is often slow, labor-intensive, costly, and error-prone. New advances in computer vision and remote sensing open possibilities to observe the Earth at a different scale. However, substantial pre-processing work is still required in order to apply state-of-the-art methodology for emergency response. To enable the comparison of methods for automatic detection of damaged buildings from post-hurricane remote sensing imagery taken from both airborne and satellite sensors…
Sustainability of the global environment is dependent on the accurate land cover information over large areas. Even with the increased number of satellite systems and sensors acquiring data with improved spectral, spatial, radiometric and temporal characteristics and the new data distribution policy, most existing land cover datasets were derived from a pixel-based single-date multi-spectral remotely sensed image with low accuracy. To improve the accuracy, the bottleneck is how to develop an accurate and effective image classification technique. By incorporating and utilizing the complete mult…
In this study, a novel machine learning algorithm is presented for disaggregation of satellite soil moisture (SM) based on self-regularized regressive models (SRRM) using high-resolution correlated information from auxiliary sources. It includes regularized clustering that assigns soft memberships to each pixel at fine-scale followed by a kernel regression that computes the value of the desired variable at all pixels. Coarse-scale remotely sensed SM were disaggregated from 10km to 1km using land cover, precipitation, land surface temperature, leaf area index, and in-situ observations of SM. Th…
Anna University（安娜大学）；Akshya College Of Engineering And Technology（阿克沙亚工程与技术学院）
Maps are used to describe far-off places . It is an aid for navigation and military strategies. Mapping of the lands are important and the mapping work is based on (i). Natural resource management & development (ii). Information technology ,(iii). Environmental development ,(iv). Facility management and (v). e-governance. The Landuse / Landcover system espoused by almost all Organisations and scientists, engineers and remote sensing community who are involved in mapping of earth surface features, is a system which is derived from the united States Geological Survey (USGS) LULC classification s…
SwinMamba: A hybrid local-global mamba framework for enhancing semantic segmentation of remotely sensed images
Xi'an Jiaotong-Liverpool University（西交利物浦大学）；University of Liverpool（利物浦大学）；Nanyang Technological University（南洋理工大学）；SJTUSME-COSMOPlat Joint Research Center for New Generation Industrial Intelligent Technology（上海交通大学机械与动力工程学院-卡奥斯新一代工业智能技术联合研究中心）
DiffSpectralNet : Unveiling the Potential of Diffusion Models for Hyperspectral Image Classification
University of Buckingham（白金汉大学）；University of Greenwich（格林威治大学）；Middlesex University（密德萨斯大学）；International University–Vietnam National University of HCMC（胡志明市越南国家大学国际大学）
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Microsoft（微软）；University of Texas at San Antonio（德克萨斯大学圣安东尼奥分校）
Remotely sensed geospatial data are critical for applications including precision agriculture, urban planning, disaster monitoring and response, and climate change research, among others. Deep learning methods are particularly promising for modeling many remote sensing tasks given the success of deep neural networks in similar computer vision tasks and the sheer volume of remotely sensed imagery available. However, the variance in data collection methods and handling of geospatial metadata make the application of deep learning methodology to remotely sensed data nontrivial. For example, satell…
Data61, CSIRO（澳大利亚联邦科学与工业研究组织数据61中心）；CSIRO Agriculture & Food（澳大利亚联邦科学与工业研究组织农业与食品研究所）；ICRAR, The University of Western Australia（西澳大利亚大学国际射电天文学研究中心）
Scene understanding of high resolution aerial images is of great importance for the task of automated monitoring in various remote sensing applications. Due to the large within-class and small between-class variance in pixel values of objects of interest, this remains a challenging task. In recent years, deep convolutional neural networks have started being used in remote sensing applications and demonstrate state of the art performance for pixel level classification of objects. \textcolor{black}{Here we propose a reliable framework for performant results for the task of semantic segmentation …
In many remote sensing applications one wants to estimate variables or parameters of interest from observations. When the target variable is available at a resolution that matches the remote sensing observations, standard algorithms such as neural networks, random forests or Gaussian processes are readily available to relate the two. However, we often encounter situations where the target variable is only available at the group level, i.e. collectively associated to a number of remotely sensed observations. This problem setting is known in statistics and machine learning as {\em multiple insta…
Wuhan University（武汉大学）；Lancaster University（兰卡斯特大学）；UK Centre for Ecology & Hydrology（英国生态与水文中心）；University of Twente（特文特大学）
Assigning geospatial objects with specific categories at the pixel level is a fundamental task in remote sensing image analysis. Along with rapid development in sensor technologies, remotely sensed images can be captured at multiple spatial resolutions (MSR) with information content manifested at different scales. Extracting information from these MSR images represents huge opportunities for enhanced feature representation and characterisation. However, MSR images suffer from two critical issues: 1) increased scale variation of geo-objects and 2) loss of detailed information at coarse spatial …
Nanjing University of Information Science and Technology（南京信息工程大学）；Beijing Electro-Mechanical Engineering Institute（北京机电工程研究所）；University of Extremadura（埃斯特雷马杜拉大学）
Convolutional neural networks (CNNs) have attracted increasing attention in the remote sensing community. Most CNNs only take the last fully-connected layers as features for the classification of remotely sensed images, discarding the other convolutional layer features which may also be helpful for classification purposes. In this paper, we propose a new adaptive deep pyramid matching (ADPM) model that takes advantage of the features from all of the convolutional layers for remote sensing image classification. To this end, the optimal fusing weights for different convolutional layers are learn…
Stanford University（斯坦福大学）；IBM Research Europe（IBM欧洲研究院）；IBM Europe（IBM欧洲）
Current methods to determine the energy efficiency of buildings require on-site visits of certified energy auditors which makes the process slow, costly, and geographically incomplete. To accelerate the identification of promising retrofit targets on a large scale, we propose to estimate building energy efficiency from widely available and remotely sensed data sources only, namely street view, aerial view, footprint, and satellite-borne land surface temperature (LST) data. After collecting data for almost 40,000 buildings in the United Kingdom, we combine these data sources by training multipl…
RS3DBench: A Comprehensive Benchmark for 3D Spatial Perception in Remote Sensing
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Software College of Zhejiang University（浙江大学软件学院）；Hangzhou City University（杭州市大学）
Pixel based algorithms including back propagation neural networks (NN) and support vector machines (SVM) have been widely used for remotely sensed image classifications. Within last few years, deep learning based image classifier like convolution neural networks (2D-CNN) are becoming popular alternatives to these classifiers. In this paper, we compare performance of patch based SVM and NN with that of a deep learning algorithms comprising of 2D-CNN and fully connected layers. Similar to CNN which utilise image patches to derive features for further classification, we propose to use patches as …
Several remote sensing software packages are used to the explicit purpose of analyzing and visualizing remotely sensed data, with the developing of remote sensing sensor technologies from last ten years. Accord-ing to literature, the remote sensing is still the lack of software tools for effective information extraction from remote sensing data. So, this paper provides a state-of-art of multi-sensor image fusion technologies as well as review on the quality evaluation of the single image or fused images in the commercial remote sensing pack-ages. It also introduces program (ALwassaiProcess) de…
A satellite image is a remotely sensed image data, where each pixel represents a specific location on earth. The pixel value recorded is the reflection radiation from the earth's surface at that location. Multispectral images are those that capture image data at specific frequencies across the electromagnetic spectrum as compared to Panchromatic images which are sensitive to all wavelength of visible light. Because of the high resolution and high dimensions of these images, they create difficulties for clustering techniques to efficiently detect clusters of different sizes, shapes and densitie…
Hamedan University of Technology（哈马丹工业大学）；Tarbiat Modares University（塔比亚特·莫达雷斯大学）
This paper proposes a spatial feature extraction method based on energy of the features for classification of the hyperspectral data. A proposed orthogonal filter set extracts spatial features with maximum energy from the principal components and then, a profile is constructed based on these features. The important characteristic of the proposed approach is that the filter sets coefficients are extracted from statistical properties of data, thus they are more consistent with the type and texture of the remotely sensed images compared with those of other filters such as Gabor. To assess the per…
The problem of supervised classification of the satellite image is considered to be the task of grouping pixels into a number of homogeneous regions in space intensity. This paper proposes a novel approach that combines a radial basic function clustering network with a growing neural gas include utility factor classifier to yield improved solutions, obtained with previous networks. The double objective technique is first used to the development of a method to perform the satellite images classification, and finally, the implementation to address the issue of the number of nodes in the hidden l…
Xi’an Jiaotong University（西安交通大学）；Xidian University（西安电子科技大学）；New York University Abu Dhabi（纽约大学阿布扎比分校）；Columbia University（哥伦比亚大学）
This paper presents a new supervised classification algorithm for remotely sensed hyperspectral image (HSI) which integrates spectral and spatial information in a unified Bayesian framework. First, we formulate the HSI classification problem from a Bayesian perspective. Then, we adopt a convolutional neural network (CNN) to learn the posterior class distributions using a patch-wise training strategy to better use the spatial information. Next, spatial information is further considered by placing a spatial smoothness prior on the labels. Finally, we iteratively update the CNN parameters using s…
