Advancing Applications of Satellite Photogrammetry: Novel Approaches for Built-up Area Modeling and Natural Environment Monitoring using Stereo/Multi-view Satellite Image-derived 3D Data
The Ohio State University（俄亥俄州立大学）
Shengxi Gui
备注：
Ph.D. Dissertation, Geospatial Data Analytics Lab, The Ohio State University, 2024, offical version is available in OhioLINK
With the development of remote sensing technology in recent decades, spaceborne sensors with sub-meter and meter spatial resolution (Worldview and PlanetScope) have achieved a considerable image quality to generate 3D geospatial data via a stereo matching pipeline. These achievements have significantly increased the data accessibility in 3D, necessitating adapting these 3D geospatial data to analyze human and natural environments. This dissertation explores several novel approaches based on stereo and multi-view satellite image-derived 3D geospatial data, to deal with remote sensing application issues for built-up area modeling and natural environment monitoring, including building model 3D reconstruction, glacier dynamics tracking, and lake algae monitoring. Specifically, the dissertation introduces four parts of novel approaches that deal with the spatial and temporal challenges with satellite-derived 3D data. The first study advances LoD-2 building modeling from satellite-derived Orthophoto and DSMs with a novel approach employing a model-driven workflow that generates building rectangular 3D geometry models. Secondly, we further enhanced our building reconstruction framework for dense urban areas and non-rectangular purposes, we implemented deep learning for unit-level segmentation and introduced a gradient-based circle reconstruction for circular buildings to develop a polygon composition technique for advanced building LoD2 reconstruction. Our third study utilizes high-spatiotemporal resolution PlanetScope satellite imagery for glacier tracking at 3D level in mid-latitude regions. Finally, we proposed a term as "Algal Behavior Function" to refine the quantification of chlorophyll-a concentrations from satellite imagery in water quality monitoring, addressing algae fluctuations and timing discrepancies between satellite observations and field measurements, thus enhancing the precision of underwater algae volume estimates. Overall, this dissertation demonstrates the extensive potential of satellite photogrammetry applications in addressing urban and environmental challenges. It further showcases innovative analytical methodologies that enhance the applicability of adapting stereo and multi-view very high-resolution satellite-derived 3D data. (See full abstract in the document)
We introduce the Satellite Neural Radiance Field (Sat-NeRF), a new end-to-end model for learning multi-view satellite photogrammetry in the wild. Sat-NeRF combines some of the latest trends in neural rendering with native satellite camera models, represented by rational polynomial coefficient (RPC) functions. The proposed method renders new views and infers surface models of similar quality to those obtained with traditional state-of-the-art stereo pipelines. Multi-date images exhibit significant changes in appearance, mainly due to varying shadows and transient objects (cars, vegetation). Rob…
Implementation of a Zed 2i Stereo Camera for High-Frequency Shoreline Change and Coastal Elevation Monitoring
College of Engineering and Computer Science, Texas A&M University–Corpus Christi（德克萨斯农工大学科珀斯克里斯蒂分校工程与计算机科学学院）；Conrad Blucher Institute for Surveying and Science（康拉德·布鲁克测绘与科学研究院）
Uncertainty Quantification Framework for Aerial and UAV Photogrammetry through Error Propagation
Geospatial Data Analytics Laboratory, The Ohio State University（地理空间数据分析实验室，俄亥俄州立大学）；Department of Civil, Environmental and Geodetic Engineering, The Ohio State University（土木、环境与大地测量工程系，俄亥俄州立大学）；Department of Electrical and Computer Engineering, The Ohio State University（电气与计算机工程系，俄亥俄州立大学）；Translational Data Analytics Institute, The Ohio State University（转化数据分析研究院，俄亥俄州立大学）
NVIDIA（英伟达）；University of Erlangen-Nuremberg（埃尔朗根-纽伦堡大学）；SenseTime（商汤科技）；Technical University of Munich（慕尼黑工业大学）
Scene reconstruction from unorganized RGB images is an important task in many computer vision applications. Multi-view Stereo (MVS) is a common solution in photogrammetry applications for the dense reconstruction of a static scene. The static scene assumption, however, limits the general applicability of MVS algorithms, as many day-to-day scenes undergo non-rigid motion, e.g., clothes, faces, or human bodies. In this paper, we open up a new challenging direction: dense 3D reconstruction of scenes with non-rigid changes observed from arbitrary, sparse, and wide-baseline views. We formulate the …
Although nowadays advanced dense image matching (DIM) algorithms are able to produce LiDAR (Light Detection And Ranging) comparable dense point clouds from satellite stereo images, the accuracy and completeness of such point clouds heavily depend on the geometric parameters of the satellite stereo images. The intersection angle between two images are normally seen as the most important one in stereo data acquisition, as the state-of-the-art DIM algorithms work best on narrow baseline (smaller intersection angle) stereos (E.g. Semi-Global Matching regards 15-25 degrees as good intersection angl…
Visibility underwater is challenging, and degrades as the distance between the subject and camera increases, making vision tasks in the forward-looking direction more difficult. We have collected underwater forward-looking stereo-vision and visual-inertial image sets in the Mediterranean and Red Sea. To our knowledge there are no other public datasets in the underwater environment acquired with this camera-sensor orientation published with ground-truth. These datasets are critical for the development of several underwater applications, including obstacle avoidance, visual odometry, 3D tracking…
GEOMAR Helmholtz Centre for Ocean Research Kiel（基尔亥姆霍兹海洋研究中心）
Reliable quantification of natural and anthropogenic gas release (e.g.\ CO$_2$, methane) from the seafloor into the water column, and potentially to the atmosphere, is a challenging task. While ship-based echo sounders such as single beam and multibeam systems allow detection of free gas, bubbles, in the water even from a great distance, exact quantification utilizing the hydroacoustic data requires additional parameters such as rise speed and bubble size distribution. Optical methods are complementary in the sense that they can provide high temporal and spatial resolution of single bubbles or…
SenseTime Research（商汤科技研究院）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University（中山大学）
Previous monocular depth estimation methods take a single view and directly regress the expected results. Though recent advances are made by applying geometrically inspired loss functions during training, the inference procedure does not explicitly impose any geometrical constraint. Therefore these models purely rely on the quality of data and the effectiveness of learning to generalize. This either leads to suboptimal results or the demand of huge amount of expensive ground truth labelled data to generate reasonable results. In this paper, we show for the first time that the monocular depth e…
The 3D documentation of the tunnel surface during construction requires fast and robust measurement systems. In the solution proposed in this paper, during tunnel advance a single camera is taking pictures of the tunnel surface from several positions. The recorded images are automatically processed to gain a 3D tunnel surface model. Image acquisition is realized by the tunneling/advance/driving personnel close to the tunnel face (= the front end of the advance). Based on the following fully automatic analysis/evaluation, a decision on the quality of the outbreak can be made within a few minute…
SDGE: Stereo Guided Depth Estimation for 360$^\circ$ Camera Sets
School of Computer Science and Technology, Harbin Institute of Technology（哈尔滨工业大学计算机科学与技术学院）；SZ DJI Technology Co., Ltd.（深圳市大疆创新科技有限公司）；The University of New South Wales（新南威尔士大学）
The reconstruction of accurate three-dimensional environment models is one of the most fundamental goals in the field of photogrammetry. Since satellite images provide suitable properties for obtaining large-scale environment reconstructions, there exist a variety of Stereo Matching based methods to reconstruct point clouds for satellite image pairs. Recently, the first Structure from Motion (SfM) based approach has been proposed, which allows to reconstruct point clouds from multiple satellite images. In this work, we propose an extension of this SfM based pipeline that allows us to reconstru…
We propose a framework that extends Blender to exploit Structure from Motion (SfM) and Multi-View Stereo (MVS) techniques for image-based modeling tasks such as sculpting or camera and motion tracking. Applying SfM allows us to determine camera motions without manually defining feature tracks or calibrating the cameras used to capture the image data. With MVS we are able to automatically compute dense scene models, which is not feasible with the built-in tools of Blender. Currently, our framework supports several state-of-the-art SfM and MVS pipelines. The modular system design enables us to i…
SurgCUT3R: Surgical Scene-Aware Continuous Understanding of Temporal 3D Representation
The Hamlyn Centre for Robotic Surgery, Imperial College London（机器人手术中心，帝国理工学院伦敦分校）；S-Lab, College of Computing and Data Science, Nanyang Technological University（S实验室， computing and Data Science学院，南洋理工大学）；Department of Computer Science, University of Liverpool（计算机科学系，利物浦大学）
3D Gaussian Splatting against Moving Objects for High-Fidelity Street Scene Reconstruction
Ontario Knowledge Intelligence Centre, Canada（加拿大安大略知识智能中心）；Canadian Institute of Interscience, Canada（加拿大跨学科科学研究所）；DeepCox Intelligence, Canada（加拿大DeepCox智能科技公司）
Segmentation-Driven Initialization for Sparse-view 3D Gaussian Splatting
Mid Sweden University（中瑞典大学）；Technical University of Berlin（柏林技术大学）；Hochschule für Technik und Wirtschaft Berlin（柏林技术与经济高等学院）；Department of Computer and Electrical Engineering, Mid Sweden University（中瑞典大学计算机与电气工程系）；Department of Telecommunication Systems, Technical University of Berlin（柏林技术大学电信系统系）
Calibration in a multi camera network has widely been studied for over several years starting from the earlier days of photogrammetry. Many authors have presented several calibration algorithms with their relative advantages and disadvantages. In a stereovision system, multiple view reconstruction is a challenging task. However, the total computational procedure in detail has not been presented before. Here in this work, we are dealing with the problem that, when a world coordinate point is fixed in space, image coordinates of that 3D point vary for different camera positions and orientations.…
The Department of Computer Science Engineering HKUST（香港科技大学计算机科学与工程系）
Multi-view Stereo (MVS) with known camera parameters is essentially a 1D search problem within a valid depth range. Recent deep learning-based MVS methods typically densely sample depth hypotheses in the depth range, and then construct prohibitively memory-consuming 3D cost volumes for depth prediction. Although coarse-to-fine sampling strategies alleviate this overhead issue to a certain extent, the efficiency of MVS is still an open challenge. In this work, we propose a novel method for highly efficient MVS that remarkably decreases the memory footprint, meanwhile clearly advancing state-of-…
Visual Computing and Computational Media Section (VCCM)（视觉计算与计算媒体部门）；College of Performance Visualization and Fine Arts (PVFA)（表演可视化与美术学院）；Department of Electrical and Computer Engineering (ECEN)（电气与计算机工程系）；Department of Computer Science and Engineering (CSCE)（计算机科学与工程系）
Estimating Fog Parameters from a Sequence of Stereo Images
Edinburgh Centre for Robotics, the School of Mathematical and Computer Sciences, Heriot-Watt University（爱丁堡机器人中心、数学与计算机科学学院、赫瑞斯泰大学）；School of Engineering and Physical Sciences, Heriot-Watt University（工程与物理科学学院、赫瑞斯泰大学）；Sense Robotics Lab, Department of Electrical and Electronic Engineering, Imperial College London（感知机器人实验室、电子与电气工程系、伦敦帝国学院）
Institute for Intelligent Systems, Esslingen University of Applied Sciences（智能系统研究所，埃斯林根应用科学大学）；Department of Computer Science, University of Freiburg（计算机科学系，弗赖堡大学）；ANDREAS STIHL AG & Co. KG（安德烈亚斯·施塔尔有限公司）
S3E: A Multi-Robot Multimodal Dataset for Collaborative SLAM
Sun Yat-sen University（中山大学）；South China Agricultural University（华南农业大学）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
