Automated driving technology has gained a lot of momentum in the last few years. For the exploration field, navigation is the important key for autonomous operation. In difficult scenarios such as snowy environment, the road is covered with snow and road detection is impossible in this situation using only basic techniques. This paper introduces detection of snowy road in forest environment using RGB camera. The method combines noise filtering technique with morphological operation to classify the image component. By using the assumption that all road is covered by snow and the snow part is defined as road area. From the perspective image of road, the vanishing point of road is one of factor to scope the region of road. This vanishing point is found with fitting triangle technique. The performance of algorithm is evaluated by two error value: False Negative Rate and False Positive Rate. The error shows that the method has high efficiency for detect road with straight road but low performance for curved road. This road region will be applied with depth information from camera to detect for obstacle in the future work.
SnowyLane: Robust Lane Detection on Snow-covered Rural Roads Using Infrastructural Elements
University of Tübingen, Faculty of Science, Department of Computer Science, Embedded Systems Group（图宾根大学科学学院计算机科学系嵌入式系统组）；Forschungszentrum Informatik (FZI) Karlsruhe（信息研究所（FZI）卡尔斯鲁厄）
In 2021, Japan recorded more than three times as much snowfall as usual, so road user maybe come across dangerous situation. The poor visibility caused by snow triggers traffic accidents. For example, 2021 January 19, due to the dry snow and the strong wind speed of 27 m / s, blizzards occurred and the outlook has been ineffective. Because of the whiteout phenomenon, multiple accidents with 17 casualties occurred, and 134 vehicles were stacked up for 10 hours over 1 km. At the night time zone, the temperature drops and the road surface tends to freeze. CCTV images on the road surface have the …
TUBITAK Space Technologies Research Institute（土耳其科学技术研究理事会空间技术研究所）
Cloud detection from remotely observed data is a critical pre-processing step for various remote sensing applications. In particular, this problem becomes even harder for RGB color images, since there is no distinct spectral pattern for clouds, which is directly separable from the Earth surface. In this paper, we adapt a deep pyramid network (DPN) to tackle this problem. For this purpose, the network is enhanced with a pre-trained parameter model at the encoder layer. Moreover, the method is able to obtain accurate pixel-level segmentation and classification results from a set of noisy labeled…
Durham University（杜伦大学）；Institute for Infocomm Research（资讯通信研究所）
Scene understanding for autonomous vehicles is a challenging computer vision task, with recent advances in convolutional neural networks (CNNs) achieving results that notably surpass prior traditional feature driven approaches. However, limited work investigates the application of such methods either within the highly unstructured off-road environment or to RGBD input data. In this work, we take an existing CNN architecture designed to perform semantic segmentation of RGB images of urban road scenes, then adapt and retrain it to perform the same task with multichannel RGBD images obtained unde…
Freespace detection is an essential component of autonomous driving technology and plays an important role in trajectory planning. In the last decade, deep learning-based free space detection methods have been proved feasible. However, these efforts were focused on urban road environments and few deep learning-based methods were specifically designed for off-road free space detection due to the lack of off-road benchmarks. In this paper, we present the ORFD dataset, which, to our knowledge, is the first off-road free space detection dataset. The dataset was collected in different scenes (woodl…
ROD: RGB-Only Fast and Efficient Off-road Freespace Detection
Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences（智能计算系统研究室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Astronomical Computing Research Center, Zhejiang Lab（天文计算研究室，浙江实验室）；Beijing Special Vehicle Academy（北京特种车辆学院）；School of Computer Science and Technology, University of Chinese Academy of Sciences（计算机科学与技术学院，中国科学院大学）
This paper presents a novel obstacle avoidance system for road robots equipped with RGB-D sensor that captures scenes of its way forward. The purpose of the system is to have road robots move around autonomously and constantly without any collision even with small obstacles, which are often missed by existing solutions. For each input RGB-D image, the system uses a new two-stage semantic segmentation network followed by the morphological processing to generate the accurate semantic map containing road and obstacles. Based on the map, the local path planning is applied to avoid possible collisi…
Dense Road Surface Grip Map Prediction from Multimodal Image Data
Finnish Geospatial Research Institute FGI, National Land Survey of Finland（芬兰地理空间研究所（FGI），芬兰国家土地测量局）；Department of Computer Science, Aalto University（阿尔托大学计算机科学系）
Road detection from the perspective of moving vehicles is a challenging issue in autonomous driving. Recently, many deep learning methods spring up for this task because they can extract high-level local features to find road regions from raw RGB data, such as Convolutional Neural Networks (CNN) and Fully Convolutional Networks (FCN). However, how to detect the boundary of road accurately is still an intractable problem. In this paper, we propose a siamesed fully convolutional networks (named as ``s-FCN-loc''), which is able to consider RGB-channel images, semantic contours and location priors…
RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsing
Frontiers Science Center for Intelligent Autonomous Systems, Tongji University（同济大学智能自主系统前沿科学中心）；The Hong Kong University of Science and Technology（香港科技大学）
The Hong Kong University of Science and Technology（香港科技大学）
The segmentation of drivable areas and road anomalies are critical capabilities to achieve autonomous navigation for robotic wheelchairs. The recent progress of semantic segmentation using deep learning techniques has presented effective results. However, the acquisition of large-scale datasets with hand-labeled ground truth is time-consuming and labor-intensive, making the deep learning-based methods often hard to implement in practice. We contribute to the solution of this problem for the task of drivable area and road anomaly segmentation by proposing a self-supervised learning approach. We…
Istituto di Matematica Applicata e Tecnologie Informatiche ‘E. Magenes’ - CNR（意大利国家研究委员会“E. 马涅斯”应用数学与信息技术研究所）；University of Chile（智利大学）；University of Science, Ho Chi Minh City（胡志明市科学大学）；Viet Nam National University, Ho Chi Minh City（胡志明市越南国家大学）；John von Neumann Institute（约翰·冯·诺依曼研究所）
This paper describes the methods submitted for evaluation to the SHREC 2022 track on pothole and crack detection in the road pavement. A total of 7 different runs for the semantic segmentation of the road surface are compared, 6 from the participants plus a baseline method. All methods exploit Deep Learning techniques and their performance is tested using the same environment (i.e.: a single Jupyter notebook). A training set, composed of 3836 semantic segmentation image/mask pairs and 797 RGB-D video clips collected with the latest depth cameras was made available to the participants. The meth…
Beijing University of Posts and Telecommunications（北京邮电大学）
The high performance of RGB-D based road segmentation methods contrasts with their rare application in commercial autonomous driving, which is owing to two reasons: 1) the prior methods cannot achieve high inference speed and high accuracy in both ways; 2) the different properties of RGB and depth data are not well-exploited, limiting the reliability of predicted road. In this paper, based on the evidence theory, an uncertainty-aware symmetric network (USNet) is proposed to achieve a trade-off between speed and accuracy by fully fusing RGB and depth data. Firstly, cross-modal feature fusion op…
Indian Institute of Technology Gandhinagar（印度理工学院甘地纳加尔分校）
In self driving car applications, there is a requirement to predict the location of the lane given an input RGB front facing image. In this paper, we propose an architecture that allows us to increase the speed and robustness of road detection without a large hit in accuracy by introducing an underlying shared feature space that is propagated over time, which serves as a flowing dynamic memory. By utilizing the gist of previous frames, we train the network to predict the current road with a greater accuracy and lesser deviation from previous frames.
The University of Adelaide（阿德莱德大学）；Northwestern Polytechnical University（西北工业大学）；Intellifusion（云天励飞）
Road detection is a critically important task for self-driving cars. By employing LiDAR data, recent works have significantly improved the accuracy of road detection. Relying on LiDAR sensors limits the wide application of those methods when only cameras are available. In this paper, we propose a novel road detection approach with RGB being the only input during inference. Specifically, we exploit pseudo-LiDAR using depth estimation, and propose a feature fusion network where RGB and learned depth information are fused for improved road detection. To further optimize the network structure and …
Towards All-Day Perception for Off-Road Driving: A Large-Scale Multispectral Dataset and Comprehensive Benchmark
Research Center for Intelligent Computing Systems, Institute of Computing Technology, Chinese Academy of Sciences, Beijing（智能计算系统研究中心，计算技术研究所，中国科学院，北京）；Beijing Institute of Control Engineering（北京控制工程研究所）
Zhejiang University（浙江大学）；Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Hangzhou SurImage Technology Co., Ltd.（杭州视芯科技有限公司）
Semantic segmentation has made striking progress due to the success of deep convolutional neural networks. Considering the demands of autonomous driving, real-time semantic segmentation has become a research hotspot these years. However, few real-time RGB-D fusion semantic segmentation studies are carried out despite readily accessible depth information nowadays. In this paper, we propose a real-time fusion semantic segmentation network termed RFNet that effectively exploits complementary cross-modal information. Building on an efficient network architecture, RFNet is capable of running swiftl…
Substantial progress has been made in the field of object detection in road scenes. However, it is mainly focused on vehicles and pedestrians. To this end, we investigate traffic cone detection, an object category crucial for road effects and maintenance. In this work, the YOLOv5 algorithm is employed, in order to find a solution for the efficient and fast detection of traffic cones. The YOLOv5 can achieve a high detection accuracy with the score of IoU up to 91.31%. The proposed method is been applied to an RGB roadwork image dataset, collected from various sources.
INRIA center at University Grenoble Alpes（法国格勒诺布尔阿尔卑斯大学INRIA中心）；Lynred（Lynred公司）；Univ Grenoble Alpes, CNRS, Grenoble INP, GIPSA-lab（法国格勒诺布尔阿尔卑斯大学、CNRS、格勒诺布尔INP、GIPSA实验室）
Accurate depth estimation under adverse night conditions has practical impact and applications, such as on autonomous driving and rescue robots. In this work, we studied monocular depth estimation at night time in which various adverse weather, light, and different road conditions exist, with data captured in both RGB and event modalities. Event camera can better capture intensity changes by virtue of its high dynamic range (HDR), which is particularly suitable to be applied at adverse night conditions in which the amount of light is limited in the scene. Although event data can retain visual …
Road transport infrastructure is critical for safe, fast, economical, and reliable mobility within the whole country that is conducive to a productive society. However, roads tend to deteriorate over time due to natural causes in the environment and repeated traffic loads. Pavement Distress (PD) detection is essential in monitoring the current conditions of the public roads to enable targeted rehabilitation and preventive maintenance. Nonetheless, distress detection surveys are still done via manual inspection for developing countries such as the Philippines. This study proposed the use of dee…
Universitat Politècnica de Catalunya（加泰罗尼亚理工大学）；Beamagine S.L.（Beamagine公司）
Detecting pedestrians is a crucial task in autonomous driving systems to ensure the safety of drivers and pedestrians. The technologies involved in these algorithms must be precise and reliable, regardless of environment conditions. Relying solely on RGB cameras may not be enough to recognize road environments in situations where cameras cannot capture scenes properly. Some approaches aim to compensate for these limitations by combining RGB cameras with TOF sensors, such as LIDARs. However, there are few works that address this problem using exclusively the 3D geometric information provided by…
TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息与电子工程学院）；Jinhua Institute of Zhejiang University（浙江大学金华研究院）；School of Computer Science, University of Nottingham Ningbo China（宁波诺丁汉大学计算机学院）
Nanjing University of Posts and Telecommunications（南京邮电大学）；Nanjing University of Science and Technology（南京理工大学）
Robust road detection is a key challenge in safe autonomous driving. Recently, with the rapid development of 3D sensors, more and more researchers are trying to fuse information across different sensors to improve the performance of road detection. Although many successful works have been achieved in this field, methods for data fusion under deep learning framework is still an open problem. In this paper, we propose a Siamese deep neural network based on FCN-8s to detect road region. Our method uses data collected from a monocular color camera and a Velodyne-64 LiDAR sensor. We project the LiD…
CSNR and JMIM Based Spectral Band Selection for Reducing Metamerism in Urban Driving
School of Engineering, University of Galway, Ireland（Galway大学工程学院，爱尔兰）；Ryan Institute, University of Galway, Ireland（Galway大学Ryan学院，爱尔兰）；Valeo Vision Systems, Tuam, Ireland（爱尔兰Tuam市Valeo Vision Systems公司）
UC San Diego（加州大学圣迭戈分校）；HKUST Robotics Institute（香港科技大学机器人研究所）；University of Bristol（布里斯托大学）
Manual visual inspection performed by certified inspectors is still the main form of road pothole detection. This process is, however, not only tedious, time-consuming and costly, but also dangerous for the inspectors. Furthermore, the road pothole detection results are always subjective, because they depend entirely on the individual experience. Our recently introduced disparity (or inverse depth) transformation algorithm allows better discrimination between damaged and undamaged road areas, and it can be easily deployed to any semantic segmentation network for better road pothole detection r…
Robotics Research Center, KCIS, IIIT Hyderabad（印度海得拉巴国际信息技术研究所 KCIS 机器人研究中心）；Montreal Institute of Learning Algorithms (MILA), Universite de Montreal（蒙特利尔大学蒙特利尔学习算法研究所）
Accurate localization of other traffic participants is a vital task in autonomous driving systems. State-of-the-art systems employ a combination of sensing modalities such as RGB cameras and LiDARs for localizing traffic participants, but most such demonstrations have been confined to plain roads. We demonstrate, to the best of our knowledge, the first results for monocular object localization and shape estimation on surfaces that do not share the same plane with the moving monocular camera. We approximate road surfaces by local planar patches and use semantic cues from vehicles in the scene t…
Normandie Univ（诺曼底大学）；INSA Rouen（鲁昂国立应用科学学院）；UNIROUEN（鲁昂大学）；UNIHAVRE（勒阿弗尔大学）；LITIS（LITIS实验室）；University of Burgundy（勃艮第大学）；UBFC（勃艮第-弗朗什孔泰大学）；ImViA（ImViA实验室）
Object detection in road scenes is necessary to develop both autonomous vehicles and driving assistance systems. Even if deep neural networks for recognition task have shown great performances using conventional images, they fail to detect objects in road scenes in complex acquisition situations. In contrast, polarization images, characterizing the light wave, can robustly describe important physical properties of the object even under poor illumination or strong reflections. This paper shows how non-conventional polarimetric imaging modality overcomes the classical methods for object detectio…
College of Intelligence and Computing, Tianjin University（天津大学智能与计算学部）
Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. It empowers smart city traffic management and disaster rescue. Researchers have made mount of efforts in this area and achieved considerable progress. Nevertheless, it is still a challenge when the objects are hard to distinguish, especially in low light conditions. To tackle this problem, we construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parki…
The RGB complementary metal-oxidesemiconductor (CMOS) sensor works within the visible light spectrum. Therefore it is very sensitive to environmental light conditions. On the contrary, a long-wave infrared (LWIR) sensor operating in 8-14 micro meter spectral band, functions independent of visible light. In this paper, we exploit both visual and thermal perception units for robust object detection purposes. After delicate synchronization and (cross-) labeling of the FLIR [1] dataset, this multi-modal perception data passes through a convolutional neural network (CNN) to detect three critical ob…
Beijing Institute of Technology（北京理工大学）；University of Bristol（布里斯托大学）；Arm（安谋科技）；ATG Robotics（ATG机器人公司）；Ryazan State Radio Engineering University（梁赞国立无线电工程大学）；Tongji University（同济大学）
This paper presents a novel pothole detection approach based on single-modal semantic segmentation. It first extracts visual features from input images using a convolutional neural network. A channel attention module then reweighs the channel features to enhance the consistency of different feature maps. Subsequently, we employ an atrous spatial pyramid pooling module (comprising of atrous convolutions in series, with progressive rates of dilation) to integrate the spatial context information. This helps better distinguish between potholes and undamaged road areas. Finally, the feature maps in…
Australian Centre for Robotic Vision（澳大利亚机器人视觉中心）；Australian National University（澳大利亚国立大学）；Queensland University of Technology（昆士兰科技大学）
Current self-driving car systems operate well in sunny weather but struggle in adverse conditions. One of the most commonly encountered adverse conditions involves water on the road caused by rain, sleet, melting snow or flooding. While some advances have been made in using conventional RGB camera and LIDAR technology for detecting water hazards, other sources of information such as polarization offer a promising and potentially superior approach to this problem in terms of performance and cost. In this paper, we present a novel stereo-polarization system for detecting and tracking water hazar…
Tongji University（同济大学）；Shanghai Research Institute for Intelligent Autonomous Systems（上海智能自主系统研究院）；the Hong Kong University of Science and Technology（香港科技大学）；SmartMore（思谋科技）；University of Thessaloniki（塞萨洛尼基大学）
Existing road pothole detection approaches can be classified as computer vision-based or machine learning-based. The former approaches typically employ 2-D image analysis/understanding or 3-D point cloud modeling and segmentation algorithms to detect road potholes from vision sensor data. The latter approaches generally address road pothole detection using convolutional neural networks (CNNs) in an end-to-end manner. However, road potholes are not necessarily ubiquitous and it is challenging to prepare a large well-annotated dataset for CNN training. In this regard, while computer vision-based…
