本文提出一种半监督学习方法，融合 Mapillary 街景图像与 OpenStreetMap 数据，通过伪标签、形态特征和楼层估算工作流自动估算建筑物高度，在德国海德堡验证中 MAE 约 2.1 米，达到领先水平。
摘要：
准确的建筑物高度估算是从新兴地理空间大数据（包括志愿地理信息 VGI）自动生成三维城市模型的关键。然而，目前仍缺少一种基于低成本 VGI 数据的大规模建筑物高度自动估算方案。VGI 数据平台的快速发展，尤其是 OpenStreetMap（OSM）和众包街景图像（SVI），为填补这一研究空白提供了令人振奋的机遇。在本工作中，我们提出了一种半监督学习（SSL）方法，利用 Mapillary SVI 和 OSM 数据自动估算建筑物高度，以低成本、开源的方式生成 LoD1 三维城市模型。所提方法由三部分组成：第一，我们提出了一种 SSL 模式，可在有监督回归过程中设置不同比例的“伪标签”；第二，我们从 OSM 数据（即建筑物和街道）中提取多层级形态测量特征，用于推断建筑物高度；第三，我们设计了一个建筑物楼层估算工作流，利用预训练的建筑立面目标检测网络从 SVI 生成“伪标签”，并将其分配给对应的 OSM 建筑物足迹。在案例研究中，我们在德国海德堡市验证了所提出的 SSL 方法，并依据建筑物高度参考数据评估了模型性能。基于三种不同的回归模型，即随机森林（RF）、支持向量机（SVM）和卷积神经网络（CNN），SSL 方法在建筑物高度估算中带来了明显的性能提升，平均绝对误差（MAE）约为 2.1 米，与当前最先进的方法相比具有竞争力。初步结果令人鼓舞，并激励我们在未来工作中基于低成本 VGI 数据扩展所提方法，并探索在数据质量和可用性各异的区域和地区应用的可能性。
英文摘要：
Accurate building height estimation is key to the automatic derivation of 3D city models from emerging big geospatial data, including Volunteered Geographical Information (VGI). However, an automatic solution for large-scale building height estimation based on low-cost VGI data is currently missing. The fast development of VGI data platforms, especially OpenStreetMap (OSM) and crowdsourced street-view images (SVI), offers a stimulating opportunity to fill this research gap. In this work, we propose a semi-supervised learning (SSL) method of automatically estimating building height from Mapillary SVI and OSM data to generate low-cost and open-source 3D city modeling in LoD1. The proposed method consists of three parts: first, we propose an SSL schema with the option of setting a different ratio of "pseudo label" during the supervised regression; second, we extract multi-level morphometric features from OSM data (i.e., buildings and streets) for the purposed of inferring building height; last, we design a building floor estimation workflow with a pre-trained facade object detection network to generate "pseudo label" from SVI and assign it to the corresponding OSM building footprint. In a case study, we validate the proposed SSL method in the city of Heidelberg, Germany and evaluate the model performance against the reference data of building heights. Based on three different regression models, namely Random Forest (RF), Support Vector Machine (SVM), and Convolutional Neural Network (CNN), the SSL method leads to a clear performance boosting in estimating building heights with a Mean Absolute Error (MAE) around 2.1 meters, which is competitive to state-of-the-art approaches. The preliminary result is promising and motivates our future work in scaling up the proposed method based on low-cost VGI data, with possibilities in even regions and areas with diverse data quality and availability.
Displaying near-real-time traffic information is a useful feature of digital navigation maps. However, most commercial providers rely on privacy-compromising measures such as deriving location information from cellphones to estimate traffic. The lack of an open-source traffic estimation method using open data platforms is a bottleneck for building sophisticated navigation services on top of OpenStreetMap (OSM). We propose a deep learning-based Complex Event Processing (CEP) method that relies on publicly available video camera streams for traffic estimation. The proposed framework performs nea…
UrbanVGGT: Scalable Sidewalk Width Estimation from Street View Images
Heinz College of Information Systems and Public Policy（信息系统与公共政策学院）；Carnegie Mellon University（卡内基梅隆大学）；Institute of Remote Sensing and Geographical Information System（遥感与地理信息系统研究所）
We propose a method that integrates two widely available data sources, building footprints from 2D maps and street level images, to derive valuable information that is generally difficult to acquire -- building heights and building facade masks in images. Building footprints are elevated in world coordinates and projected onto images. Building heights are estimated by scoring projected footprints based on their alignment with building features in images. Building footprints with estimated heights can be converted to simple 3D building models, which are projected back to images to identify buil…
Building height estimation is important in many applications such as 3D city reconstruction, urban planning, and navigation. Recently, a new building height estimation method using street scene images and 2D maps was proposed. This method is more scalable than traditional methods that use high-resolution optical data, LiDAR data, or RADAR data which are expensive to obtain. The method needs to detect building rooflines and then compute building height via the pinhole camera model. We observe that this method has limitations in handling complex street scene images in which buildings overlap wit…
Technical University of Munich (TUM)（慕尼黑工业大学）；German Aerospace Center (DLR)（德国航空航天中心）
Land-use classification based on spaceborne or aerial remote sensing images has been extensively studied over the past decades. Such classification is usually a patch-wise or pixel-wise labeling over the whole image. But for many applications, such as urban population density mapping or urban utility planning, a classification map based on individual buildings is much more informative. However, such semantic classification still poses some fundamental challenges, for example, how to retrieve fine boundaries of individual buildings. In this paper, we proposed a general framework for classifying…
Royal Melbourne Institute of Technology（皇家墨尔本理工大学）
On-road bicycle lanes improve safety for cyclists, and encourage participation in cycling for active transport and recreation. With many local authorities responsible for portions of the infrastructure, official maps and datasets of bicycle lanes may be out-of-date and incomplete. Even "crowdsourced" databases may have significant gaps, especially outside popular metropolitan areas. This thesis presents a method to create a map of bicycle lanes in a survey area by taking sample street scene images from each road, and then applying a deep learning model that has been trained to recognise bicycl…
Norwegian University of Science and Technology（挪威科技大学）；Wuhan University（武汉大学）；Nanjing University of Finance & Economics（南京财经大学）
Navigation services utilized by autonomous vehicles or ordinary users require the availability of detailed information about road-related objects and their geolocations, especially at road intersections. However, these road intersections are mainly represented as point elements without detailed information, or are even not available in current versions of crowdsourced mapping databases including OpenStreetMap(OSM). This study develops an approach to automatically detect road objects and place them to right location from street-level images. Our processing pipeline relies on two convolutional n…
Chinese University of Hong Kong（香港中文大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）
The ability to recognize the position and order of the floor-level lines that divide adjacent building floors can benefit many applications, for example, urban augmented reality (AR). This work tackles the problem of locating floor-level lines in street-view images, using a supervised deep learning approach. Unfortunately, very little data is available for training such a network $-$ current street-view datasets contain either semantic annotations that lack geometric attributes, or rectified facades without perspective priors. To address this issue, we first compile a new dataset and develop a…
BuildingView: Constructing Urban Building Exteriors Databases with Street View Imagery and Multimodal Large Language Mode
Thrust of Urban Governance and Design, The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州）城市治理与设计中心）；Spatial Sciences Institute, University of Southern California（南加州大学空间科学学院）
We have developed a framework for crisis response and management that incorporates the latest technologies in computer vision (CV), inland flood prediction, damage assessment and data visualization. The framework uses data collected before, during, and after the crisis to enable rapid and informed decision making during all phases of disaster response. Our computer-vision model analyzes spaceborne and airborne imagery to detect relevant features during and after a natural disaster and creates metadata that is transformed into actionable information through web-accessible mapping tools. In part…
GLObal Building heights for Urban Studies (UT-GLOBUS) for city- and street- scale urban simulations: Development and first applications
University of Texas at Austin（德克萨斯大学奥斯汀分校）；Indian Institute of Tropical Meteorology（印度热带气象研究所）；CIEMAT（西班牙国家研究中心（能源、环境与技术研究中心））；Purdue University（普渡大学）；National Center for Atmospheric Research（美国国家大气研究中心）
OpenFACADES: An Open Framework for Architectural Caption and Attribute Data Enrichment via Street View Imagery
Department of Architecture, National University of Singapore（建筑系，新加坡国立大学）；Department of Electrical and Computer Engineering, National University of Singapore（电气与计算机工程系，新加坡国立大学）；School of Artificial Intelligence, Shenzhen Technology University（人工智能学院，深圳科技大学）；Department of Real Estate, National University of Singapore（房地产系，新加坡国立大学）
Examining the Commitments and Difficulties Inherent in Multimodal Foundation Models for Street View Imagery
Guanghua School of Management, Peking University（北京大学光华管理学院）；The Bartlett School of Sustainable Construction, University College London（伦敦大学学院巴特lett可持续建筑学院）；School of Architecture, Tsinghua University（清华大学建筑学院）；School of Computing, The University of Georgia（佐治亚大学计算学院）；Department of Geography, University College London（伦敦大学学院地理系）；Department of Geography and the Environment, University of Texas at Austin（德克萨斯大学奥斯汀分校地理与环境系）
Australian National University（澳大利亚国立大学）；University of Oxford（牛津大学）；University of Technology Sydney（悉尼科技大学）
This paper presents a new approach for synthesizing a novel street-view panorama given an overhead satellite image. Taking a small satellite image patch as input, our method generates a Google's omnidirectional street-view type panorama, as if it is captured from the same geographical location as the center of the satellite patch. Existing works tackle this task as an image generation problem which adopts generative adversarial networks to implicitly learn the cross-view transformations, while ignoring the domain relevance. In this paper, we propose to explicitly establish the geometric corres…
Sun Yat-sen University（中山大学）；Peking University（北京大学）；The Chinese University of Hong Kong（香港中文大学）；SenseTime Research（商汤科技研究院）；Wuhan University（武汉大学）
This paper presents OmniCity, a new dataset for omnipotent city understanding from multi-level and multi-view images. More precisely, the OmniCity contains multi-view satellite images as well as street-level panorama and mono-view images, constituting over 100K pixel-wise annotated images that are well-aligned and collected from 25K geo-locations in New York City. To alleviate the substantial pixel-wise annotation efforts, we propose an efficient street-view image annotation pipeline that leverages the existing label maps of satellite view and the transformation relations between different vie…
Correctly identifying crosswalks is an essential task for the driving activity and mobility autonomy. Many crosswalk classification, detection and localization systems have been proposed in the literature over the years. These systems use different perspectives to tackle the crosswalk classification problem: satellite imagery, cockpit view (from the top of a car or behind the windshield), and pedestrian perspective. Most of the works in the literature are designed and evaluated using small and local datasets, i.e. datasets that present low diversity. Scaling to large datasets imposes a challen…
Georgia Institute of Technology（佐治亚理工学院）；Virginia Tech（弗吉尼亚理工学院暨州立大学）；University of Maryland at College Park（马里兰大学帕克分校）
Cross-view matching refers to the problem of finding the closest match for a given query ground view image to one from a database of aerial images. If the aerial images are geotagged, then the closest matching aerial image can be used to localize the query ground view image. Due to the recent success of deep learning methods, several cross-view matching techniques have been proposed. These approaches perform well for the matching of isolated query images. However, their evaluation over a trajectory is limited. In this paper, we evaluate cross-view matching for the task of localizing a ground v…
Sat2City v2: Native 3D City Asset Generation from a Single Satellite Image
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Chinese University of Hong Kong（香港中文大学）；Auckland University of Technology（奥克兰理工大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（香港科技大学计算机科学与工程系）
