An Object-Based Deep Learning Approach for Building Height Estimation from Single SAR Images
Department of Electrical, Computer and Biomedical Engineering, University of Pavia（电子、计算机与生物医学工程系，帕维亚大学）；Department of Engineering, University of Sannio（工程系，萨尼奥大学）
Babak Memar, Luigi Russo, Silvia Liberata Ullo, Paolo Gamba
Accurate estimation of building heights using very high resolution (VHR) synthetic aperture radar (SAR) imagery is crucial for various urban applications. This paper introduces a Deep Learning (DL)-based methodology for automated building height estimation from single VHR COSMO-SkyMed images: an object-based regression approach based on bounding box detection followed by height estimation. This model was trained and evaluated on a unique multi-continental dataset comprising eight geographically diverse cities across Europe, North and South America, and Asia, employing a cross-validation strategy to explicitly assess out-of-distribution (OOD) generalization. The results demonstrate highly promising performance, particularly on European cities where the model achieves a Mean Absolute Error (MAE) of approximately one building story (2.20 m in Munich), significantly outperforming recent state-of-the-art methods in similar OOD scenarios. Despite the increased variability observed when generalizing to cities in other continents, particularly in Asia with its distinct urban typologies and prevalence of high-rise structures, this study underscores the significant potential of DL for robust cross-city and cross-continental transfer learning in building height estimation from single VHR SAR data.
OceanSAR-2: A Universal Feature Extractor for SAR Ocean Observation
Galeio, Paris, France（法国加利欧）；LOPS, Ifremer, Plouzané, France（法国若特默尔实验室）；OceanScope, Brest, France（法国海洋观测中心）；Nanjing University of Information Science and Technology, Nanjing, China（中国信息科学技术大学）
3D-SAR Tomography and Machine Learning for High-Resolution Tree Height Estimation
University of Cambridge（剑桥大学）；United Nations Office for Outer Space Affairs（联合国外层空间事务厅）；UK Department for Energy Security and Net Zero（英国能源安全与净零部）；INSEC TEC；Drexel University（卓克索大学）
Tomographic SAR Reconstruction for Forest Height Estimation
University of Cambridge（剑桥大学）；United Nations Office for Outer Space Affairs（联合国外层空间事务厅）；UK Department for Energy Security and Net Zero（英国能源安全与净零部）；INSEC TEC；Drexel University（卓克索大学）
本文提出直接由二维 SLC SAR 图像利用深度学习估计森林冠层高度，并评估 SLC 数量影响；结果表明绕过完整层析处理会使误差增加 16%–21%。
Originally developed in fields such as robotics and autonomous driving with image-based navigation in mind, deep learning-based single-image depth estimation (SIDE) has found great interest in the wider image analysis community. Remote sensing is no exception, as the possibility to estimate height maps from single aerial or satellite imagery bears great potential in the context of topographic reconstruction. A few pioneering investigations have demonstrated the general feasibility of single image height prediction from optical remote sensing images and motivate further studies in that directio…
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心）
Building height retrieval from synthetic aperture radar (SAR) imagery is of great importance for urban applications, yet highly challenging owing to the complexity of SAR data. This paper addresses the issue of building height retrieval in large-scale urban areas from a single TerraSAR-X spotlight or stripmap image. Based on the radar viewing geometry, we propose that this problem can be formulated as a bounding box regression problem and therefore allows for integrating height data from multiple data sources in generating ground truth on a larger scale. We introduce building footprints from g…
Technische Universität München（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）
Multi-baseline interferometric synthetic aperture radar (InSAR) techniques are effective approaches for retrieving the 3-D information of urban areas. In order to obtain a plausible reconstruction, it is necessary to use more than twenty interferograms. Hence, these methods are commonly not appropriate for large-scale 3-D urban mapping using TanDEM-X data where only a few acquisitions are available in average for each city. This work proposes a new SAR tomographic processing framework to work with those extremely small stacks, which integrates the non-local filtering into SAR tomography invers…
A Deep Learning Approach to Estimate Canopy Height and Uncertainty by Integrating Seasonal Optical, SAR and Limited GEDI LiDAR Data over Northern Forests
Building height estimation is important in many applications such as 3D city reconstruction, urban planning, and navigation. Recently, a new building height estimation method using street scene images and 2D maps was proposed. This method is more scalable than traditional methods that use high-resolution optical data, LiDAR data, or RADAR data which are expensive to obtain. The method needs to detect building rooflines and then compute building height via the pinhole camera model. We observe that this method has limitations in handling complex street scene images in which buildings overlap wit…
Technical University of Munich（慕尼黑工业大学）；German Aerospace Center（德国航空航天中心）
Monocular height estimation (MHE) from remote sensing imagery has high potential in generating 3D city models efficiently for a quick response to natural disasters. Most existing works pursue higher performance. However, there is little research exploring the interpretability of MHE networks. In this paper, we target at exploring how deep neural networks predict height from a single monocular image. Towards a comprehensive understanding of MHE networks, we propose to interpret them from multiple levels: 1) Neurons: unit-level dissection. Exploring the semantic and height selectivity of the lea…
Evaluating the effects of preprocessing, method selection, and hyperparameter tuning on SAR-based flood mapping and water depth estimation
EDF R&D（EDF研究与发展）；Laboratoire National d’Hydraulique et Environnement (LNHE)（国家水利与环境实验室）；ENPC（巴黎综合理工学院）；Institut Polytechnique de Paris（巴黎理工学院）；Inria（法国国家信息与自动化技术研究所）
SHLE: Devices Tracking and Depth Filtering for Stereo-based Height Limit Estimation
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Department of Management Science and Engineering, Tsinghua University（清华大学管理科学与工程系）
Simon Fraser University（西蒙弗雷泽大学）；University of British Columbia（不列颠哥伦比亚大学）
Drug repurposing can accelerate the identification of effective compounds for clinical use against SARS-CoV-2, with the advantage of pre-existing clinical safety data and an established supply chain. RNA viruses such as SARS-CoV-2 manipulate cellular pathways and induce reorganization of subcellular structures to support their life cycle. These morphological changes can be quantified using bioimaging techniques. In this work, we developed DEEMD: a computational pipeline using deep neural network models within a multiple instance learning framework, to identify putative treatments effective aga…
Microsoft（微软）；Welthungerhilfe（德国世界饥饿援助组织）；GRID Council（GRID委员会）；Dr. B C Roy PGI PS（比·西·罗伊博士儿科研究生学院）
Malnutrition is a global health crisis and is the leading cause of death among children under five. Detecting malnutrition requires anthropometric measurements of weight, height, and middle-upper arm circumference. However, measuring them accurately is a challenge, especially in the global south, due to limited resources. In this work, we propose a CNN-based approach to estimate the height of standing children under five years from depth images collected using a smart-phone. According to the SMART Methodology Manual [5], the acceptable accuracy for height is less than 1.4 cm. On training our d…
University of Alberta（阿尔伯塔大学）；vGeomatics Inc.（3v地理信息公司）
Over the past decade, Interferometric Synthetic Aperture Radar (InSAR) has become a successful remote sensing technique. However, during the acquisition step, microwave reflections received at satellite are usually disturbed by strong noise, leading to a noisy single-look complex (SLC) SAR image. The quality of their interferometric phase is even worse. InSAR phase filtering is an ill-posed problem and plays a key role in subsequent processing. However, most of existing methods usually require expert supervision or heavy runtime, which limits the usability and scalability for practical usages …
Understanding the 3D geometric structure of the Earth's surface has been an active research topic in photogrammetry and remote sensing community for decades, serving as an essential building block for various applications such as 3D digital city modeling, change detection, and city management. Previous researches have extensively studied the problem of height estimation from aerial images based on stereo or multi-view image matching. These methods require two or more images from different perspectives to reconstruct 3D coordinates with camera information provided. In this paper, we deal with t…
University of Houston（休斯顿大学）；University of Ioannina（约阿尼纳大学）
In this paper, we propose a novel regression-based method for employing privileged information to estimate the height using human metrology. The actual values of the anthropometric measurements are difficult to estimate accurately using state-of-the-art computer vision algorithms. Hence, we use ratios of anthropometric measurements as features. Since many anthropometric measurements are not available at test time in real-life scenarios, we employ a learning using privileged information (LUPI) framework in a regression setup. Instead of using the LUPI paradigm for regression in its original for…
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院空天信息创新研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing University of Posts and Telecommunications（北京邮电大学）
Florida Southern College（佛罗里达南方学院）；University of Central Florida（中佛罗里达大学）
Egocentric, or first-person vision which became popular in recent years with an emerge in wearable technology, is different than exocentric (third-person) vision in some distinguishable ways, one of which being that the camera wearer is generally not visible in the video frames. Recent work has been done on action and object recognition in egocentric videos, as well as work on biometric extraction from first-person videos. Height estimation can be a useful feature for both soft-biometrics and object tracking. Here, we propose a method of estimating the height of an egocentric camera without an…
Remote Sensing Technology Institute (IMF), German Aerospace Center (DLR)（德国航空航天中心遥感技术研究所（IMF））；Signal Processing in Earth Observation (SiPEO), Technical University of Munich (TUM)（慕尼黑工业大学地球观测信号处理研究所（SiPEO））
In this paper we tackle a very novel problem, namely height estimation from a single monocular remote sensing image, which is inherently ambiguous, and a technically ill-posed problem, with a large source of uncertainty coming from the overall scale. We propose a fully convolutional-deconvolutional network architecture being trained end-to-end, encompassing residual learning, to model the ambiguous mapping between monocular remote sensing images and height maps. Specifically, it is composed of two parts, i.e., convolutional sub-network and deconvolutional sub-network. The former corresponds to…
We propose a method that integrates two widely available data sources, building footprints from 2D maps and street level images, to derive valuable information that is generally difficult to acquire -- building heights and building facade masks in images. Building footprints are elevated in world coordinates and projected onto images. Building heights are estimated by scoring projected footprints based on their alignment with building features in images. Building footprints with estimated heights can be converted to simple 3D building models, which are projected back to images to identify buil…
In this paper, a new probability density function (pdf) is proposed to model the statistics of wavelet coefficients, and a simple Kalman's filter is derived from the new pdf using Bayesian estimation theory. Specifically, we decompose the speckled image into wavelet subbands, we apply the Kalman's filter to the high subbands, and reconstruct a despeckled image from the modified detail coefficients. Experimental results demonstrate that our method compares favorably to several other despeckling methods on test synthetic aperture radar (SAR) images.
Enhancing Monocular Height Estimation via Weak Supervision from Imperfect Labels
Chair of Data Science in Earth Observation, Technical University of Munich (TUM)（地球观测数据科学教授职位，慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；School of Engineering and Design, Technical University of Munich (TUM)（工程与设计学院，慕尼黑技术大学）
TSE-Net: Semi-supervised Monocular Height Estimation from Single Remote Sensing Images
Chair of Data Science in Earth Observation, Technical University of Munich (TUM)（地球观测数据科学教授职位，慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）
A machine learning approach for image classification in synthetic aperture RADAR
Department of Mathematics and Statistics University of Limerick（数学与统计学系利默里克大学）；Health Research Institute (HRI)（健康研究中心）；Department of Computer Science and Information Systems University of Limerick（计算机科学与信息系统系利默里克大学）
The First Assessment of PhiSat-2 Imagery for Monocular Building Height Estimation
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University（信息工程测绘遥感国家重点实验室（LIESMARS），武汉大学）；Department of Geomatics Engineering, Institut Teknologi Sepuluh Nopember（工程技术学院测绘工程系，印度尼西亚技术学院十月份十号学院）；Department of Informatics and Information Technology, College of Natural and Applied Sciences (CoNAS), Sokoine University of Agriculture（信息与信息技术系，索科因农业学院自然与应用科学学院）；Geomatics Department, Tshwane University of Technology（测绘系，塔什韦恩技术大学）；Department of Geography, Environmental Management and Energy Studies, Faculty of Science, University of Johannesburg（地理、环境管理与能源研究系，科学学院，约翰内斯堡大学）
Given a single scene image, this paper proposes a method of Category-level 6D Object Pose and Size Estimation (COPSE) from the point cloud of the target object, without external real pose-annotated training data. Specifically, beyond the visual cues in RGB images, we rely on the shape information predominately from the depth (D) channel. The key idea is to explore the shape alignment of each instance against its corresponding category-level template shape, and the symmetric correspondence of each object category for estimating a coarse 3D object shape. Our framework deforms the point cloud of …
Deep learning based landslide density estimation on SAR data for rapid response
University of California Berkeley（加州大学伯克利分校）；The Ohio State University（俄亥俄州立大学）；German Aerospace Center DLR（德国航空航天中心（DLR））；University of Valencia（瓦伦西亚大学）；National Observatory of Athens（雅典国家天文台）；University of Oxford（牛津大学）；NVIDIA（英伟达）；Universidad de Antioquia（安蒂奥基亚大学）
Imperial College London（伦敦帝国理工学院）；Hammersmith Hospital（哈默史密斯医院）；Imperial College Healthcare NHS Trust（帝国理工学院医疗保健国民保健服务信托基金）
Objective: To demonstrate the effectiveness of using a deep learning-based approach for a fully automated slice-based measurement of muscle mass for assessing sarcopenia on CT scans of the abdomen without any case exclusion criteria. Materials and Methods: This retrospective study was conducted using a collection of public and privately available CT images (n = 1070). The method consisted of two stages: slice detection from a CT volume and single-slice CT segmentation. Both stages used Fully Convolutional Neural Networks (FCNN) and were based on a UNet-like architecture. Input data consisted o…
The Chinese University of Hong Kong（香港中文大学）；Sensetime Group Limited（商汤集团有限公司）
Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images. Since the location recovery in 3D space is quite difficult on account of absence of depth information, this paper proposes a novel unified framework which decomposes the detection problem into a structured polygon prediction task and a depth recovery task. Different from the widely studied 2D bounding boxes, the proposed novel structured polygon in the 2D image consists of several projected surfaces of the target object. Compared to the widely-used 3D bounding box proposals, it is…
BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images
Graduate School of Engineering, Ozyegin university（奥济根大学工程研究生院）；Department of Computer Science, University of Rochester（罗切斯特大学计算机科学系）；School of Engineering and Natural Sciences, Istanbul Medipol University（伊斯坦布尔梅迪波尔大学工程与自然科学学院）；Huawei Turkiye R&D Center（华为土耳其研发中心）
