从卫星栅格影像中对城市区域进行3D重建,长期以来一直是学术和工业研究中一个具有挑战性的目标。如今,能够在细节层次2(Level Of Details 2,LOD2)实现这一目标的方法较为稀少,它们依赖于基于几何的过程化方法,并需要立体影像和/或LIDAR数据作为输入。本文提出一种名为KIBS(Keypoints Inference By Segmentation,基于分割的关键点推理)的城市3D重建方法,该方法包含两个新颖特征:i)用于屋顶截面3D检测的完全深度学习方法;ii)仅将单张(非正射)卫星栅格影像作为模型输入。这通过两个步骤实现:i)由一个Mask R-CNN模型对建筑屋顶截面执行2D分割,并将这些分割出的像素融合到RGB卫星栅格影像中;ii)由另一个相同的Mask R-CNN模型通过全景分割推理屋顶截面角点距地面的高度,进而实现建筑和城市的完整3D重建。我们通过在数分钟内重建不同城市区域展示了KIBS方法的潜力:在两个数据集上,单个屋顶截面2D分割的Jaccard指数分别为88.55%和75.21%;在两个数据集上,此类正确分割像素用于3D重建的高度平均误差分别为1.60 m和2.06 m,因此处于LOD2精度范围内。
英文摘要:
Reconstructing urban areas in 3D out of satellite raster images has been a long-standing and challenging goal of both academical and industrial research. The rare methods today achieving this objective at a Level Of Details $2$ rely on procedural approaches based on geometry, and need stereo images and/or LIDAR data as input. We here propose a method for urban 3D reconstruction named KIBS(\textit{Keypoints Inference By Segmentation}), which comprises two novel features: i) a full deep learning approach for the 3D detection of the roof sections, and ii) only one single (non-orthogonal) satellite raster image as model input. This is achieved in two steps: i) by a Mask R-CNN model performing a 2D segmentation of the buildings' roof sections, and after blending these latter segmented pixels within the RGB satellite raster image, ii) by another identical Mask R-CNN model inferring the heights-to-ground of the roof sections' corners via panoptic segmentation, unto full 3D reconstruction of the buildings and city. We demonstrate the potential of the KIBS method by reconstructing different urban areas in a few minutes, with a Jaccard index for the 2D segmentation of individual roof sections of $88.55\%$ and $75.21\%$ on our two data sets resp., and a height's mean error of such correctly segmented pixels for the 3D reconstruction of $1.60$ m and $2.06$ m on our two data sets resp., hence within the LOD2 precision range.
Predicting the Solar Potential of Rooftops using Image Segmentation and Structured Data
namR(namR公司);ENSTA Paris(巴黎高等先进技术学院);Ecole polytechnique(巴黎综合理工学院);IP Paris(巴黎理工学院);Université de Paris(巴黎大学);Ecole normale supérieure(巴黎高等师范学院);Université Paris Dauphine(巴黎多芬大学)
Progressively Growing Generative Adversarial Networks for High Resolution Semantic Segmentation of Satellite Images
Louisiana State University(路易斯安那州立大学);Northeastern University(东北大学);NASA Ames Research Center/BAERI(美国国家航空航天局艾姆斯研究中心/BAERI);BAER Institute(BAER研究所);NASA Ames Research Center(美国国家航空航天局艾姆斯研究中心)
ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction
School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences(中国科学院大学电子电气与通信工程学院);College of Computing, Department of Data Science, City University of Hong Kong(香港城市大学计算学院数据科学系);Aerospace Information Research Institute, Chinese Academy of Sciences(中国科学院空天信息创新研究院);Department of Electrical and Computer Engineering, The University of Hong Kong(香港大学电气与计算机工程系)
Multi-task deep learning for large-scale building detail extraction from high-resolution satellite imagery
Nanjing Normal University(南京师范大学);Peking University(北京大学);Technical University of Munich(慕尼黑工业大学);Shenyang Jianzhu University(沈阳建筑大学);Nanjing University(南京大学)
Urban Surface Reconstruction in SAR Tomography by Graph-Cuts
Télécom ParisTech, Université Paris-Saclay(巴黎萨克雷大学电信学院);Univ Lyon, UJM-Saint-Etienne(里昂大学,圣艾蒂安大学);ONERA, The French Aerospace Laboratory(法国航空航天研究院);Institute of Electronics and Telecommunications of Rennes (IETR), University of Rennes 1(雷恩第一大学雷恩电子与电信研究所)
Deep Learning for Pavement Condition Evaluation Using Satellite Imagery
Department of Civil and Environmental Engineering, University of Houston(土木与环境工程系,休斯顿大学);College of Business, North Dakota State University(商学院,北达科他州立大学);Department of Civil and Environmental Engineering, University of South Florida(土木与环境工程系,佛罗里达州立大学)