Sensing population distribution from satellite imagery via deep learning: model selection, neighboring effect, and systematic biases
University of Arkansas（阿肯色大学）；University of Minnesota（明尼苏达大学）；Massachusetts Institute of Technology（麻省理工学院）；Michigan Technological University（密歇根理工大学）；Texas A&M University（德克萨斯A&M大学）
Xiao Huang, Di Zhu, Fan Zhang, Tao Liu, Xiao Li, Lei Zou
The rapid development of remote sensing techniques provides rich, large-coverage, and high-temporal information of the ground, which can be coupled with the emerging deep learning approaches that enable latent features and hidden geographical patterns to be extracted. This study marks the first attempt to cross-compare performances of popular state-of-the-art deep learning models in estimating population distribution from remote sensing images, investigate the contribution of neighboring effect, and explore the potential systematic population estimation biases. We conduct an end-to-end training of four popular deep learning architectures, i.e., VGG, ResNet, Xception, and DenseNet, by establishing a mapping between Sentinel-2 image patches and their corresponding population count from the LandScan population grid. The results reveal that DenseNet outperforms the other three models, while VGG has the worst performances in all evaluating metrics under all selected neighboring scenarios. As for the neighboring effect, contradicting existing studies, our results suggest that the increase of neighboring sizes leads to reduced population estimation performance, which is found universal for all four selected models in all evaluating metrics. In addition, there exists a notable, universal bias that all selected deep learning models tend to overestimate sparsely populated image patches and underestimate densely populated image patches, regardless of neighboring sizes. The methodological, experimental, and contextual knowledge this study provides is expected to benefit a wide range of future studies that estimate population distribution via remote sensing imagery.
Hunan University（湖南大学）；State Key Laboratory of Integrated Services Networks, Xidian University（西安电子科技大学综合业务网理论及关键技术国家重点实验室）；Peng Cheng Laboratory（鹏城实验室）；Australian National University（澳大利亚国立大学）；University of Macau（澳门大学）；Sichuan University（四川大学）；Helmholtz-Zentrum Dresden-Rossendorf（亥姆霍兹德累斯顿-罗森多夫研究中心）；Helmholtz Institute Freiberg for Resource Technology（亥姆霍兹弗莱贝格资源技术研究所）
The use of deep learning for water extraction requires precise pixel-level labels. However, it is very difficult to label high-resolution remote sensing images at the pixel level. Therefore, we study how to utilize point labels to extract water bodies and propose a novel method called the neighbor feature aggregation network (NFANet). Compared with pixellevel labels, point labels are much easier to obtain, but they will lose much information. In this paper, we take advantage of the similarity between the adjacent pixels of a local water-body, and propose a neighbor sampler to resample remote s…
Deep networks have recently enjoyed enormous success when applied to recognition and classification problems in computer vision, but their use in graphics problems has been limited. In this work, we present a novel deep architecture that performs new view synthesis directly from pixels, trained from a large number of posed image sets. In contrast to traditional approaches which consist of multiple complex stages of processing, each of which require careful tuning and can fail in unexpected ways, our system is trained end-to-end. The pixels from neighboring views of a scene are presented to the…
TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models
University of British Columbia（不列颠哥伦比亚大学）；Vector Institute for AI（Vector人工智能研究院）；Toyota Technological Institute at Chicago（芝加哥丰田理工学院）；Carleton College（卡尔顿学院）；University of Pennsylvania（宾夕法尼亚大学）
Accurate and robust detection of multi-class objects in optical remote sensing images is essential to many real-world applications such as urban planning, traffic control, searching and rescuing, etc. However, state-of-the-art object detection techniques designed for images captured using ground-level sensors usually experience a sharp performance drop when directly applied to remote sensing images, largely due to the object appearance differences in remote sensing images in term of sparse texture, low contrast, arbitrary orientations, large scale variations, etc. This paper presents a novel o…
University of Warwick（华威大学）；NYU（纽约大学）；The Alan Turing Institute（艾伦·图灵研究所）
We propose a generalizable framework for the population estimation of dense, informal settlements in low-income urban areas--so called 'slums'--using high-resolution satellite imagery. Precise population estimates are a crucial factor for efficient resource allocations by government authorities and NGO's, for instance in medical emergencies. We utilize equitable ground-truth data, which is gathered in collaboration with local communities: Through training and community mapping, the local population contributes their unique domain knowledge, while also maintaining agency over their data. This p…
The human visual system is composed of multiple physiological components that apply multiple mechanisms in order to cope with the rich visual content it encounters. The complexity of this system leads to non-trivial relations between what we see and what we perceive, and in particular, between the raw intensities of an image that we display and the ones we perceive where various visual biases and illusions are introduced. In this paper we describe a method for reducing a large class of biases related to the lateral inhibition mechanism in the human retina where neurons suppress the activity of…
The application of infrared hyperspectral imagery to geological problems is becoming more popular as data become more accessible and cost-effective. Clustering and classifying spectrally similar materials is often a first step in applications ranging from economic mineral exploration on Earth to planetary exploration on Mars. Semi-manual classification guided by expertly developed spectral parameters can be time consuming and biased, while supervised methods require abundant labeled data and can be difficult to generalize. Here we develop a fully unsupervised workflow for feature extraction an…
SPyCer: Semi-Supervised Physics-Guided Contextual Attention for Near-Surface Air Temperature Estimation from Satellite Imagery
INSA CVL, Université d’Orléans, PRISME UR 4229（INSA CVL，奥尔良大学，PRISME UR 4229）；Université d’Orléans, INSA CVL, PRISME UR 4229（奥尔良大学，INSA CVL，PRISME UR 4229）；Université d’Orléans, CEDETE, UR 1210（奥尔良大学，CEDETE，UR 1210）
Spatially Aware Deep Learning for Microclimate Prediction from High-Resolution Geospatial Imagery
Tel Aviv University, Faculty of Life Sciences, School of Zoology（特拉维夫大学，生命科学学院，动物学系）；Tel Aviv University, Center for Artificial Intelligence & Data Science（特拉维夫大学，人工智能与数据科学中心）；School of BioSciences, The University of Melbourne（墨尔本大学生物科学学院）
Forest canopy height estimation from satellite RGB imagery using large-scale airborne LiDAR-derived training data and monocular depth estimation
Beijing Normal University（北京师范大学）；CSIRO Environment（英联邦科学与工业研究组织环境部）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；Wuhan University（武汉大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）
Politecnico di Torino（都灵理工大学）；Politecnico Interdepartmental Centre for Service Robotics（都灵理工大学跨部门服务机器人中心）；Università degli Studi di Torino（都灵大学）；Institute of Electronics, Computer and Telecommunication Engineering of the National Research Council of Italy（意大利国家研究委员会电子、计算机与电信工程研究所）
Precision agriculture is considered to be a fundamental approach in pursuing a low-input, high-efficiency, and sustainable kind of agriculture when performing site-specific management practices. To achieve this objective, a reliable and updated description of the local status of crops is required. Remote sensing, and in particular satellite-based imagery, proved to be a valuable tool in crop mapping, monitoring, and diseases assessment. However, freely available satellite imagery with low or moderate resolutions showed some limits in specific agricultural applications, e.g., where crops are gr…
Seabed-Net: A multi-task network for joint bathymetry estimation and seabed classification from remote sensing imagery in shallow waters
Faculty of Electrical Engineering and Computer Science, Technische Universität Berlin（技术大学柏林电子工程与计算机科学学院）；BIFOLD - Berlin Institute for the Foundations of Learning and Data（柏林学习与数据基础研究所）
DivShift: Exploring Domain-Specific Distribution Shifts in Large-Scale, Volunteer-Collected Biodiversity Datasets
Stanford University（斯坦福大学）；University of Freiburg（弗莱堡大学）；Carnegie Science（卡内基科学研究所）；Federal University of Minas Gerais（米纳斯吉拉斯联邦大学）；University of California, Berkeley（加州大学伯克利分校）；Howard Hughes Medical Institute（霍华德·休斯医学研究所）
Deep learning has become in recent years a cornerstone tool fueling key innovations in the industry, such as autonomous driving. To attain good performances, the neural network architecture used for a given application must be chosen with care. These architectures are often handcrafted and therefore prone to human biases and sub-optimal selection. Neural Architecture Search (NAS) is a framework introduced to mitigate such risks by jointly optimizing the network architectures and its weights. Albeit its novelty, it was applied on complex tasks with significant results - e.g. semantic image segm…
This paper introduces the use of single layer and deep convolutional networks for remote sensing data analysis. Direct application to multi- and hyper-spectral imagery of supervised (shallow or deep) convolutional networks is very challenging given the high input data dimensionality and the relatively small amount of available labeled data. Therefore, we propose the use of greedy layer-wise unsupervised pre-training coupled with a highly efficient algorithm for unsupervised learning of sparse features. The algorithm is rooted on sparse representations and enforces both population and lifetime …
College of Microelectronics and Communication Engineering, Chongqing University（重庆大学微电子与通信工程学院）；Chongqing Key Laboratory of Space Information Network and Intelligent Information Fusion（重庆空间信息网络与智能信息融合重点实验室）；Lancaster Environment Centre, Lancaster University（兰卡斯特大学兰卡斯特环境中心）；UK Centre for Ecology & Hydrology（英国生态与水文中心）
Change detection is one of the fundamental applications of synthetic aperture radar (SAR) images. However, speckle noise presented in SAR images has a much negative effect on change detection. In this research, a novel two-phase object-based deep learning approach is proposed for multi-temporal SAR image change detection. Compared with traditional methods, the proposed approach brings two main innovations. One is to classify all pixels into three categories rather than two categories: unchanged pixels, changed pixels caused by strong speckle (false changes), and changed pixels formed by real t…
FLNet: Flood-Induced Agriculture Damage Assessment using Super Resolution of Satellite Images
Annam.AI CoE, MoE, Indian Institute of Technology Ropar, Punjab, India（Annam.AI联合研究所、教育部、印度理工学院罗帕尔分校）；Department of Mathematical Sciences, Rajiv Gandhi Institute of Petroleum Technology, Jais, UP, India（数学科学系、拉贾吉安石油技术学院）；Department of Computer Science and Engineering, Indian Institute of Technology Ropar, Punjab, India（计算机科学与工程系、印度理工学院罗帕尔分校）
Finnish Environment Institute（芬兰环境研究所）；Metsähallitus, National Parks（芬兰林业管理局国家公园）
Remote sensing benefits habitat conservation by making monitoring of large areas easier compared to field surveying especially if the remote sensed data can be automatically analyzed. An important aspect of monitoring is classifying and mapping habitat types present in the monitored area. Automatic classification is a difficult task, as classes have fine-grained differences and their distributions are long-tailed and unbalanced. Usually training data used for automatic land cover classification relies on fully annotated segmentation maps, annotated from remote sensed imagery to a fairly high-l…
From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models
University of South Florida Tampa USA；University of Alabama Tuscaloosa USA；University of Michigan Ann Arbor USA；Texas Tech University Lubbock USA；Texas A \& M University College Station USA；University of Pittsburgh Pittsburgh USA；University of South Florida；University of Alabama；University of Michigan；Texas Tech University；Texas A \& M University；University of Pittsburgh
