Road network extraction from satellite images is widely applicated in intelligent traffic management and autonomous driving fields. The high-resolution remote sensing images contain complex road areas and distracted background, which make it a challenge for road extraction. In this study, we present a stacked multitask network for end-to-end segmenting roads while preserving connectivity correctness. In the network, a global-aware module is introduced to enhance pixel-level road feature representation and eliminate background distraction from overhead images; a road-direction-related connectivity task is added to ensure that the network preserves the graph-level relationships of the road segments. We also develop a stacked multihead structure to jointly learn and effectively utilize the mutual information between connectivity learning and segmentation learning. We evaluate the performance of the proposed network on three public remote sensing datasets. The experimental results demonstrate that the network outperforms the state-of-the-art methods in terms of road segmentation accuracy and connectivity maintenance.
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Road curb detection is important for autonomous driving. It can be used to determine road boundaries to constrain vehicles on roads, so that potential accidents could be avoided. Most of the current methods detect road curbs online using vehicle-mounted sensors, such as cameras or 3-D Lidars. However, these methods usually suffer from severe occlusion issues. Especially in highly-dynamic traffic environments, most of the field of view is occupied by dynamic objects. To alleviate this issue, we detect road curbs offline using high-resolution aerial images in this paper. Moreover, the detected r…
Fine-Grained Traffic Inference from Road to Lane via Spatio-Temporal Graph Node Generation
Fudan University（复旦大学）；Zhuhai Fudan Innovation Research Institute（珠海复旦创新研究院）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Guangzhou University（广州大学）；Shanghai Jiao Tong University（上海交通大学）
Road extraction is an essential step in building autonomous navigation systems. Detecting road segments is challenging as they are of varying widths, bifurcated throughout the image, and are often occluded by terrain, cloud, or other weather conditions. Using just convolution neural networks (ConvNets) for this problem is not effective as it is inefficient at capturing distant dependencies between road segments in the image which is essential to extract road connectivity. To this end, we propose a Spatial and Interaction Space Graph Reasoning (SPIN) module which when plugged into a ConvNet per…
École Polytechnique Fédérale de Lausanne（洛桑联邦理工学院）；Stanford University（斯坦福大学）
We propose a novel, connectivity-oriented loss function for training deep convolutional networks to reconstruct network-like structures, like roads and irrigation canals, from aerial images. The main idea behind our loss is to express the connectivity of roads, or canals, in terms of disconnections that they create between background regions of the image. In simple terms, a gap in the predicted road causes two background regions, that lie on the opposite sides of a ground truth road, to touch in prediction. Our loss function is designed to prevent such unwanted connections between background r…
The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Road-boundary detection is important for autonomous driving. It can be used to constrain autonomous vehicles running on road areas to ensure driving safety. Compared with online road-boundary detection using on-vehicle cameras/Lidars, offline detection using aerial images could alleviate the severe occlusion issue. Moreover, the offline detection results can be directly employed to annotate high-definition (HD) maps. In recent years, deep-learning technologies have been used in offline detection. But there still lacks a publicly available dataset for this task, which hinders the research progr…
Peking University（北京大学）；China North Vehicle Research Institute（中国北方车辆研究所）
Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. The problem has been mainly formulated in early works as a binary classification one, e.g. associating pixels with road or non-road labels. Whereas understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse, and the various mechanical performance of off-road robots may lead to different definitions of safe regions to traverse. How to define and annotate fine-grained labels to achieve meaningful scene understanding for a robot t…
RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
Department of Electronic Engineering, BNRist, Tsinghua University（清华大学电子工程系、北京信息科学与技术国家研究中心）；Zhongguancun Academy（中关村学院）；Amap, Alibaba Group（阿里巴巴集团高德地图）
Peking University（北京大学）；China North Vehicle Research Institute（中国北方车辆研究所）
Off-road semantic segmentation with fine-grained labels is necessary for autonomous vehicles to understand driving scenes, as the coarse-grained road detection can not satisfy off-road vehicles with various mechanical properties. Fine-grained semantic segmentation in off-road scenes usually has no unified category definition due to ambiguous nature environments, and the cost of pixel-wise labeling is extremely high. Furthermore, semantic properties of off-road scenes can be very changeable due to various precipitations, temperature, defoliation, etc. To address these challenges, this research …
A Large-Scale Remote Sensing Dataset and VLM-based Algorithm for Fine-Grained Road Hierarchy Classification
School of Geospatial Engineering and Science（地理工程与科学学院）；Laboratory of Intelligent Collaborative Computing（智能协同计算实验室）；United Nations University Institute in Macau（联合国澳门研究所）；Moganshan Geospatial Information Laboratory（莫干山地理信息实验室）
The previous fine-grained datasets mainly focus on classification and are often captured in a controlled setup, with the camera focusing on the objects. We introduce the first Fine-Grained Vehicle Detection (FGVD) dataset in the wild, captured from a moving camera mounted on a car. It contains 5502 scene images with 210 unique fine-grained labels of multiple vehicle types organized in a three-level hierarchy. While previous classification datasets also include makes for different kinds of cars, the FGVD dataset introduces new class labels for categorizing two-wheelers, autorickshaws, and truck…
Perelman School of Medicine, University of Pennsylvania（宾夕法尼亚大学佩雷尔曼医学院）
Brain age prediction based on neuroimaging data could help characterize both the typical brain development and neuropsychiatric disorders. Pattern recognition models built upon functional connectivity (FC) measures derived from resting state fMRI (rsfMRI) data have been successfully used to predict the brain age. However, most existing studies focus on coarse-grained FC measures between brain regions or intrinsic connectivity networks (ICNs), which may sacrifice fine-grained FC information of the rsfMRI data. Whole brain voxel-wise FC measures could provide fine-grained FC information of the b…
Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；ETH Zürich（苏黎世联邦理工学院）
This paper tackles the task of estimating the topology of road networks from aerial images. Building on top of a global model that performs a dense semantical classification of the pixels of the image, we design a Convolutional Neural Network (CNN) that predicts the local connectivity among the central pixel of an input patch and its border points. By iterating this local connectivity we sweep the whole image and infer the global topology of the road network, inspired by a human delineating a complex network with the tip of their finger. We perform an extensive and comprehensive qualitative an…
Towards a safe and comfortable driving, road scene segmentation is a rudimentary problem in camera-based advance driver assistance systems (ADAS). Despite of the great achievement of Convolutional Neural Networks (CNN) for semantic segmentation task, the high computational efforts of CNN based methods is still a challenging area. In recent work, we proposed a novel approach to utilise the advantages of CNNs for the task of road segmentation at reasonable computational effort. The runtime benefits from using irregular super pixels as basis for the input for the CNN rather than the image grid, w…
Automatic crack detection on pavement surfaces is an important research field in the scope of developing an intelligent transportation infrastructure system. In this paper, a cost effective solution for road crack inspection by mounting commercial grade sport camera, GoPro, on the rear of the moving vehicle is introduced. Also, a novel method called ConnCrack combining conditional Wasserstein generative adversarial network and connectivity maps is proposed for road crack detection. In this method, a 121-layer densely connected neural network with deconvolution layers for multi-level feature fu…
Semantic Change Detection of Roads and Bridges: A Fine-grained Dataset and Multimodal Frequency-driven Detector
MOE Key Lab of ICSP, IMIS Lab of Anhui Province, Anhui Provincial Key Lab of Multimodal Cognitive Computation, School of Computer Science and Technology, Anhui University, Hefei, China（教育部ICSP重点实验室、安徽IMIS实验室、安徽省多模态认知计算重点实验室、计算机科学与技术学院、安徽大学，合肥，中国）；School of Public Safety and Emergency Management, Anhui University of Science and Technology, Hefei, China（公共安全与应急管理学院、安徽理工大学，合肥，中国）
RoadGIE: Towards A Global-Scale Aerial Benchmark for Generalizable Interactive Road Extraction
NKIARI, Shenzhen Futian（深圳福田NKIARI）；VCIP, CS, Nankai University（南开大学VCIP研究所）；AAIS, Nankai University（南开大学AAIS）；College of Electronic Engineering, National University of Defense Technology, Changsha, China（国防科技大学电子工程学院，长沙，中国）
URoadNet: Dual Sparse Attentive U-Net for Multiscale Road Network Extraction
College of Automation & College of Artificial Intelligence, Nanjing University of Posts and Telecommunications（南京邮电大学自动化学院与人工智能学院）；School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；Jarvis Research Center, Tencent YouTu Lab（腾讯优图实验室贾维斯研究中心）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Qatar Computing Research Institute, HBKU（卡塔尔计算研究所，哈马德·本·哈利法大学）
Mapping road networks is currently both expensive and labor-intensive. High-resolution aerial imagery provides a promising avenue to automatically infer a road network. Prior work uses convolutional neural networks (CNNs) to detect which pixels belong to a road (segmentation), and then uses complex post-processing heuristics to infer graph connectivity. We show that these segmentation methods have high error rates because noisy CNN outputs are difficult to correct. We propose RoadTracer, a new method to automatically construct accurate road network maps from aerial images. RoadTracer uses an i…
Central Queensland University（中央昆士兰大学）；Department of Transport and Main Roads（交通与主干道部）
Accurate estimation of the biomass of roadside grasses plays a significant role in applications such as fire-prone region identification. Current solutions heavily depend on field surveys, remote sensing measurements and image processing using reference markers, which often demand big investments of time, effort and cost. This paper proposes Density Weighted Connectivity of Grass Pixels (DWCGP) to automatically estimate grass biomass from roadside image data. The DWCGP calculates the length of continuously connected grass pixels along a vertical orientation in each image column, and then weigh…
Universitat Oberta de Catalunya（加泰罗尼亚开放大学）；Computer Vision Lab, ETH Zürich（苏黎世联邦理工学院计算机视觉实验室）
This paper tackles the task of estimating the topology of filamentary networks such as retinal vessels and road networks. Building on top of a global model that performs a dense semantical classification of the pixels of the image, we design a Convolutional Neural Network (CNN) that predicts the local connectivity between the central pixel of an input patch and its border points. By iterating this local connectivity we sweep the whole image and infer the global topology of the filamentary network, inspired by a human delineating a complex network with the tip of their finger. We perform an ext…
Classifying and counting vehicles in road traffic has numerous applications in the transportation engineering domain. However, the wide variety of vehicles (two-wheelers, three-wheelers, cars, buses, trucks etc.) plying on roads of developing regions without any lane discipline, makes vehicle classification and counting a hard problem to automate. In this paper, we use state of the art Convolutional Neural Network (CNN) based object detection models and train them for multiple vehicle classes using data from Delhi roads. We get upto 75% MAP on an 80-20 train-test split using 5562 video frames …
Automatically extracting roads from satellite imagery is a fundamental yet challenging computer vision task in the field of remote sensing. Pixel-wise semantic segmentation-based approaches and graph-based approaches are two prevailing schemes. However, prior works show the imperfections that semantic segmentation-based approaches yield road graphs with low connectivity, while graph-based methods with iterative exploring paradigms and smaller receptive fields focus more on local information and are also time-consuming. In this paper, we propose a new scheme for multi-task satellite imagery roa…
Most state-of-the-art approaches to road extraction from aerial images rely on a CNN trained to label road pixels as foreground and remainder of the image as background. The CNN is usually trained by minimizing pixel-wise losses, which is less than ideal to produce binary masks that preserve the road network's global connectivity. To address this issue, we introduce an Adversarial Learning (AL) strategy tailored for our purposes. A naive one would treat the segmentation network as a generator and would feed its output along with ground-truth segmentations to a discriminator. It would then trai…
We propose an automated method to estimate a road segment's free-flow speed from overhead imagery and road metadata. The free-flow speed of a road segment is the average observed vehicle speed in ideal conditions, without congestion or adverse weather. Standard practice for estimating free-flow speeds depends on several road attributes, including grade, curve, and width of the right of way. Unfortunately, many of these fine-grained labels are not always readily available and are costly to manually annotate. To compensate, our model uses a small, easy to obtain subset of road features along wit…
University of Kentucky（肯塔基大学）；Kentucky Transportation Center（肯塔基交通中心）
This paper addresses the task of road safety assessment. An emerging approach for conducting such assessments in the United States is through the US Road Assessment Program (usRAP), which rates roads from highest risk (1 star) to lowest (5 stars). Obtaining these ratings requires manual, fine-grained labeling of roadway features in street-level panoramas, a slow and costly process. We propose to automate this process using a deep convolutional neural network that directly estimates the star rating from a street-level panorama, requiring milliseconds per image at test time. Our network also est…
SVII-3D: Advancing Roadside Infrastructure Inventory with Decimeter-level 3D Localization and Comprehension from Sparse Street Imagery
State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing (LIESMARS), Wuhan University, Wuhan 430079, China（信息工程测绘遥感国家重点实验室（LIESMARS），武汉大学）；Research Institute Ltd, Chengdu 610000, China（四川省公路规划设计研究有限公司）
The binary segmentation of roads in very high resolution (VHR) remote sensing images (RSIs) has always been a challenging task due to factors such as occlusions (caused by shadows, trees, buildings, etc.) and the intra-class variances of road surfaces. The wide use of convolutional neural networks (CNNs) has greatly improved the segmentation accuracy and made the task end-to-end trainable. However, there are still margins to improve in terms of the completeness and connectivity of the results. In this paper, we consider the specific context of road extraction and present a direction-aware resi…
Automatic Extraction of road network from satellite images is a goal that can benefit and even enable new technologies. Methods that combine machine learning (ML) and computer vision have been proposed in recent years which make the task semi-automatic by requiring the user to provide curated training samples. The process can be fully automatized if training samples can be produced algorithmically. Of course, this requires a robust algorithm that can reconstruct the road networks from satellite images reliably so that the output can be fed as training samples. In this work, we develop such a t…
Beyond Endpoints: Path-Centric Reasoning for Vectorized Off-Road Network Extraction
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences（中国科学院大学杭州高等研究院）
Leveraging the Spatial Hierarchy: Coarse-to-fine Trajectory Generation via Cascaded Hybrid Diffusion
Singapore-MIT Alliance for Research and Technology（新加坡-麻省理工联合研究技术联盟）；Rutgers University（罗格斯大学）；University of Florida（佛罗里达大学）；Massachusetts Institute of Technology（麻省理工学院）
Traffic light detection is essential for self-driving cars to navigate safely in urban areas. Publicly available traffic light datasets are inadequate for the development of algorithms for detecting distant traffic lights that provide important navigation information. We introduce a novel benchmark traffic light dataset captured using a synchronized pair of narrow-angle and wide-angle cameras covering urban and semi-urban roads. We provide 1032 images for training and 813 synchronized image pairs for testing. Additionally, we provide synchronized video pairs for qualitative analysis. The datas…
SignRAG: A Retrieval-Augmented System for Scalable Zero-Shot Road Sign Recognition
Computer Engineering The Ohio State University（计算机工程 俄亥俄州立大学）；Applied Research Transportation Research Center Inc.（应用研究 交通运输研究中心公司）；Computer Engineering The Ohio State University Columbus, Ohio, USA（计算机工程 俄亥俄州立大学 哥伦布俄亥俄州）；Applied Research Transportation Research Center Inc. East Liberty, Ohio, USA（应用研究 交通运输研究中心公司 埃斯特利比 俄亥俄州）
