Accurate local-level poverty measurement is an essential task for governments and humanitarian organizations to track the progress towards improving livelihoods and distribute scarce resources. Recent computer vision advances in using satellite imagery to predict poverty have shown increasing accuracy, but they do not generate features that are interpretable to policymakers, inhibiting adoption by practitioners. Here we demonstrate an interpretable computational framework to accurately predict poverty at a local level by applying object detectors to high resolution (30cm) satellite images. Using the weighted counts of objects as features, we achieve 0.539 Pearson's r^2 in predicting village-level poverty in Uganda, a 31% improvement over existing (and less interpretable) benchmarks. Feature importance and ablation analysis reveal intuitive relationships between object counts and poverty predictions. Our results suggest that interpretability does not have to come at the cost of performance, at least in this important domain.
The combination of high-resolution satellite imagery and machine learning have proven useful in many sustainability-related tasks, including poverty prediction, infrastructure measurement, and forest monitoring. However, the accuracy afforded by high-resolution imagery comes at a cost, as such imagery is extremely expensive to purchase at scale. This creates a substantial hurdle to the efficient scaling and widespread adoption of high-resolution-based approaches. To reduce acquisition costs while maintaining accuracy, we propose a reinforcement learning approach in which free low-resolution im…
Mission San Jose High School（米慎圣何塞高中）；Stanford Univeristy（斯坦福大学）
Determining the poverty levels of various regions throughout the world is crucial in identifying interventions for poverty reduction initiatives and directing resources fairly. However, reliable data on global economic livelihoods is hard to come by, especially for areas in the developing world, hampering efforts to both deploy services and monitor/evaluate progress. This is largely due to the fact that this data is obtained from traditional door-to-door surveys, which are time consuming and expensive. Overhead satellite imagery contain characteristics that make it possible to estimate the reg…
Leveraging ChatGPT's Multimodal Vision Capabilities to Rank Satellite Images by Poverty Level: Advancing Tools for Social Science Research
Center for Applied Intelligent Systems Research (CAISR), Halmstad University, Sweden（应用智能系统研究中心（CAISR），哈马斯特德大学，瑞典）；Department of Human Geography, Lund University, Sweden（人类地理系，吕勒奥大学，瑞典）；Department of Earth and Environmental Sciences, Lund University, Sweden（地球与环境科学系，吕勒奥大学，瑞典）
Obtaining reliable data describing local poverty metrics at a granularity that is informative to policy-makers requires expensive and logistically difficult surveys, particularly in the developing world. Not surprisingly, the poverty stricken regions are also the ones which have a high probability of being a war zone, have poor infrastructure and sometimes have governments that do not cooperate with internationally funded development efforts. We train a CNN on free and publicly available daytime satellite images of the African continent from Landsat 7 to build a model for predicting local econ…
Obtaining detailed and reliable data about local economic livelihoods in developing countries is expensive, and data are consequently scarce. Previous work has shown that it is possible to measure local-level economic livelihoods using high-resolution satellite imagery. However, such imagery is relatively expensive to acquire, often not updated frequently, and is mainly available for recent years. We train CNN models on free and publicly available multispectral daytime satellite images of the African continent from the Landsat 7 satellite, which has collected imagery with global coverage for a…
KidSat: satellite imagery to map childhood poverty dataset and benchmark
University of Oxford（牛津大学）；University College London（伦敦大学学院）；National University of Singapore（新加坡国立大学）；University of Copenhagen（哥本哈根大学）；Imperial College London（帝国理工学院）；Save the Children, UK（救助儿童会英国分会）
A Satellite Imagery Dataset for Long-Term Sustainable Development in United States Cities
University of Helsinki（赫尔辛基大学）；Beijing National Research Center for Information Science and Technology (BNRist)（北京信息科学与技术国家研究中心（BNRist））；Tsinghua University（清华大学）
Lancaster University（兰卡斯特大学）；European Space Agency（欧洲空间局）
Earth observation is fundamental for a range of human activities including flood response as it offers vital information to decision makers. Semantic segmentation plays a key role in mapping the raw hyper-spectral data coming from the satellites into a human understandable form assigning class labels to each pixel. In this paper, we introduce a prototype-based interpretable deep semantic segmentation (IDSS) method, which is highly accurate as well as interpretable. Its parameters are in orders of magnitude less than the number of parameters used by deep networks such as U-Net and are clearly i…
The lack of reliable data in developing countries is a major obstacle to sustainable development, food security, and disaster relief. Poverty data, for example, is typically scarce, sparse in coverage, and labor-intensive to obtain. Remote sensing data such as high-resolution satellite imagery, on the other hand, is becoming increasingly available and inexpensive. Unfortunately, such data is highly unstructured and currently no techniques exist to automatically extract useful insights to inform policy decisions and help direct humanitarian efforts. We propose a novel machine learning approach …
Building coverage statistics provide crucial insights into the urbanization, infrastructure, and poverty level of a region, facilitating efforts towards alleviating poverty, building sustainable cities, and allocating infrastructure investments and public service provision. Global mapping of buildings has been made more efficient with the incorporation of deep learning models into the pipeline. However, these models typically rely on high-resolution satellite imagery which are expensive to collect and infrequently updated. As a result, building coverage data are not updated timely especially i…
A satellite foundation model for improved wealth monitoring
Department of Computer Science, Stanford University（斯坦福大学计算机科学系）；Department of Earth System Science, Stanford University（斯坦福大学地球系统科学系）；Center on Food Security and the Environment, Stanford University（斯坦福大学食品安全与环境中心）；Development Economics Data Group, World Bank Group（世界银行发展经济学数据组）；Department of Environmental Social Science, Stanford University（斯坦福大学环境社会科学系）
Knowledge-Guided Machine Learning: Illustrating the use of Explainable Boosting Machines to Identify Overshooting Tops in Satellite Imagery
The Courant Institute of Mathematical Sciences, New York University（Courant数学科学研究所，纽约大学）；Department of Statistics, Colorado State University（统计学系，科罗拉多州立大学）；Cooperative Institute for Research in the Atmosphere, Colorado State University（大气研究合作研究所，科罗拉多州立大学）；Department of Electrical and Computer Engineering, Colorado State University（电气与计算机工程系，科罗拉多州立大学）；Department of Mathematics, Colorado State University（数学系，科罗拉多州立大学）；School of Earth and Environmental Sciences, Seoul National University（地球与环境科学学院，首尔国立大学）
Beloit College（贝洛伊特学院）；SSEC, University of Wisconsin, Madison（威斯康星大学麦迪逊分校空间科学与工程中心）；Stanford University（斯坦福大学）
Wildfire forecasting has been one of the most critical tasks that humanities want to thrive. It plays a vital role in protecting human life. Wildfire prediction, on the other hand, is difficult because of its stochastic and chaotic properties. We tackled the problem by interpreting a series of wildfire images as a video and used it to anticipate how the fire would behave in the future. However, creating video prediction models that account for the inherent uncertainty of the future is challenging. The bulk of published attempts is based on stochastic image-autoregressive recurrent networks, wh…
Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach
LIGM, Ecole des Ponts, Univ Gustave Eiffel, CNRS（LIGM实验室、巴黎路桥学校、古斯塔夫·埃菲尔大学、法国国家科学研究中心）；Inria Paris（巴黎信息与自动化研究所）；Ecole normale superieure (ENS-PSL, CNRS, Inria)（巴黎高等师范学院（PSL大学成员）、法国国家科学研究中心、法国信息与自动化研究所）；Center for Data Science, New York University（纽约大学数据科学中心）
KANO: Kolmogorov-Arnold Neural Operator for Image Super-Resolution
Southeast University（东南大学）；Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航空信息研究所）；College of Resources and Environment, University of Chinese Academy of Sciences（中国科学院大学资源与环境学院）；Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK（格勒诺布尔阿尔卑斯大学、法国国家科研机构（Inria）、法国国家科学研究中心（CNRS）、格勒诺布尔INP、LJK）
EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation
School of Electronics and Information Engineering, Harbin Institute of Technology（电子与信息工程学院，哈尔滨工业大学）；Faculty of Computing, Harbin Institute of Technology（计算机学院，哈尔滨工业大学）；Peng Cheng Lab（鹏城实验室）
Univ Lyon（里昂大学）；Inria（法国国家信息与自动化研究所）；CNRS（法国国家科学研究中心）；ENS de Lyon（里昂高等师范学院）；Université Claude Bernard Lyon 1（里昂第一大学克洛德·贝尔纳大学）；Central European University（中欧大学）
Urbanisation is a great challenge for modern societies, promising better access to economic opportunities while widening socioeconomic inequalities. Accurately tracking how this process unfolds has been challenging for traditional data collection methods, while remote sensing information offers an alternative to gather a more complete view on these societal changes. By feeding a neural network with satellite images one may recover the socioeconomic information associated to that area, however these models lack to explain how visual features contained in a sample, trigger a given prediction. He…
