Recently, self-driving vehicles have been introduced with several automated features including lane-keep assistance, queuing assistance in traffic-jam, parking assistance and crash avoidance. These self-driving vehicles and intelligent visual traffic surveillance systems mainly depend on cameras and sensors fusion systems. Adverse weather conditions such as heavy fog, rain, snow, and sandstorms are considered dangerous restrictions of the functionality of cameras impacting seriously the performance of adopted computer vision algorithms for scene understanding (i.e., vehicle detection, tracking, and recognition in traffic scenes). For example, reflection coming from rain flow and ice over roads could cause massive detection errors which will affect the performance of intelligent visual traffic systems. Additionally, scene understanding and vehicle detection algorithms are mostly evaluated using datasets contain certain types of synthetic images plus a few real-world images. Thus, it is uncertain how these algorithms would perform on unclear images acquired in the wild and how the progress of these algorithms is standardized in the field. To this end, we present a new dataset (benchmark) consisting of real-world images collected under various adverse weather conditions called DAWN. This dataset emphasizes a diverse traffic environment (urban, highway and freeway) as well as a rich variety of traffic flow. The DAWN dataset comprises a collection of 1000 images from real-traffic environments, which are divided into four sets of weather conditions: fog, snow, rain and sandstorms. The dataset is annotated with object bounding boxes for autonomous driving and video surveillance scenarios. This data helps interpreting effects caused by the adverse weather conditions on the performance of vehicle detection systems.
National Key Laboratory of Science and Technology on Multispectral Information Processing, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院多谱信息处理技术国家级重点实验室）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）
Optical flow estimation has made great progress, but usually suffers from degradation under adverse weather. Although semi/full-supervised methods have made good attempts, the domain shift between the synthetic and real adverse weather images would deteriorate their performance. To alleviate this issue, our start point is to unsupervisedly transfer the knowledge from source clean domain to target degraded domain. Our key insight is that adverse weather does not change the intrinsic optical flow of the scene, but causes a significant difference for the warp error between clean and degraded imag…
Removing adverse weather conditions like rain, fog, and snow from images is an important problem in many applications. Most methods proposed in the literature have been designed to deal with just removing one type of degradation. Recently, a CNN-based method using neural architecture search (All-in-One) was proposed to remove all the weather conditions at once. However, it has a large number of parameters as it uses multiple encoders to cater to each weather removal task and still has scope for improvement in its performance. In this work, we focus on developing an efficient solution for the a…
National Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（华中科技大学人工智能与自动化学院多光谱信息智能处理技术国家重点实验室）；Huawei International Co. Ltd.（华为国际有限公司）；School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；School of Electrical Engineering and Computer Science, Peking University（北京大学电气与计算机工程学院）
Fourier Domain Adaptation for Traffic Light Detection in Adverse Weather
Manipal Institute of Technology（曼尼帕尔理工学院）；Manipal Academy of Higher Education（曼尼帕尔高等教育学院）；Wells Fargo International Solutions Private Limited（富国国际解决方案私人有限公司）
UMDATrack: Unified Multi-Domain Adaptive Tracking Under Adverse Weather Conditions
Beijing University of Posts and Telecommunications（北京邮电大学）；Sun Yat-sen University（中山大学）；University of Chinese Academy of Sciences（中国科学院大学）；MoE Key Laboratory of Information Technology（教育部信息科学技术重点实验室）；Guangdong Key Laboratory of Information Security Technology（广东省信息安全技术重点实验室）
RoSe: Robust Self-supervised Stereo Matching under Adverse Weather Conditions
Department of Computer Science, City University of Hong Kong（城市大学计算机科学系）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所（CASIA））
Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure
School of Telecommunications Engineering, Xidian University（西安电子科技大学通信工程学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；University of New South Wales (UNSW)（新南威尔士大学）；Shandong Normal University（山东师范大学）
Lancaster University（兰卡斯特大学）；Universidade Federal de Minas Gerais（米纳斯吉拉斯联邦大学）
Video stabilization plays a central role to improve videos quality. However, despite the substantial progress made by these methods, they were, mainly, tested under standard weather and lighting conditions, and may perform poorly under adverse conditions. In this paper, we propose a synthetic-aware adverse weather robust algorithm for video stabilization that does not require real data and can be trained only on synthetic data. We also present Silver, a novel rendering engine to generate the required training data with an automatic ground-truth extraction procedure. Our approach uses our speci…
Autonomous cars are an emergent technology which has the capacity to change human lives. The current sensor systems which are most capable of perception are based on optical sensors. For example, deep neural networks show outstanding results in recognising objects when used to process data from cameras and Light Detection And Ranging (LiDAR) sensors. However these sensors perform poorly under adverse weather conditions such as rain, fog, and snow due to the sensor wavelengths. This thesis explores new sensing developments based on long wave polarised infrared (IR) imagery and imaging radar to …
Institute of Sensors, Signals and Systems, Heriot-Watt University（赫瑞-瓦特大学传感器、信号与系统研究所）
Datasets for autonomous cars are essential for the development and benchmarking of perception systems. However, most existing datasets are captured with camera and LiDAR sensors in good weather conditions. In this paper, we present the RAdar Dataset In Adverse weaThEr (RADIATE), aiming to facilitate research on object detection, tracking and scene understanding using radar sensing for safe autonomous driving. RADIATE includes 3 hours of annotated radar images with more than 200K labelled road actors in total, on average about 4.6 instances per radar image. It covers 8 different categories of a…
Despite the recent advances of deep neural networks, object detection for adverse weather remains challenging due to the poor perception of some sensors in adverse weather. Instead of relying on one single sensor, multimodal fusion has been one promising approach to provide redundant detection information based on multiple sensors. However, most existing multimodal fusion approaches are ineffective in adjusting the focus of different sensors under varying detection environments in dynamic adverse weather conditions. Moreover, it is critical to simultaneously observe local and global informatio…
Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；University of Cambridge（剑桥大学）；Institute of High Performance Computing, Agency for Science, Technology and Research（新加坡科技研究局高性能计算研究所）；Tianjin University（天津大学）；Hong Kong Metropolitan University（香港都会大学）
We presented a method for improving computer vision tasks on images affected by adverse weather conditions, including distortions caused by adherent raindrops. Overcoming the challenge of applying computer vision to images affected by adverse weather conditions is essential for autonomous vehicles utilizing RGB cameras. For this purpose, we trained an appropriate generative adversarial network and showed that it was effective at removing the effect of the distortions, in the context of image reconstruction and computer vision tasks. We showed that object recognition, a vital task for autonomou…
SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal
School of Computer Science and Cyber Engineering, Guangzhou University（广州大学计算机科学与网络工程学院）；College of Engineering, Huaqiao University（华侨大学工学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络空间安全学院）
Uncertainty-Guided Adverse Weather Restoration via Gated Transformer Network
School of Automation, Southeast University（东南大学自动化学院）；Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich（慕尼黑工业大学机器人、人工智能与实时系统讲席教授组）
Unifying Physically-Informed Weather Priors in A Single Model for Image Restoration Across Multiple Adverse Weather Conditions
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China（香港中文大学（深圳）计算机科学与工程系）；Shanghai Artificial Intelligence Laboratory, Shanghai, China（上海人工智能实验室）；ROAS Thrust, the Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China and The Hong Kong University of Science and Technology, Department of Electronic and Computer Engineering, Hong Kong SAR, China（香港科学与技术大学（广州）ROAS方向及电子与计算机工程系，香港特别行政区）
3D Semantic Segmentation in the Wild: Learning Generalized Models for Adverse-Condition Point Clouds
Nanyang Technological University（南洋理工大学）；Waseda University（早稻田大学）；Technical University of Denmark（丹麦技术大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Ottawa（渥太华大学）；Carnegie Mellon University（卡内基梅隆大学）
Segmenting each moving object instance in a scene is essential for many applications. But like many other computer vision tasks, this task performs well in optimal weather, but then adverse weather tends to fail. To be robust in weather conditions, the usual way is to train network in data of given weather pattern or to fuse multiple sensors. We focus on a new possibility, that is, to improve its resilience to weather interference through the network's structural design. First, we propose a novel FPN structure called RiWFPN with a progressive top-down interaction and attention refinement modul…
Robust Single Object Tracking in LiDAR Point Clouds under Adverse Weather Conditions
School of Mathematical Sciences, Dalian University of Technology（大连理工大学数学科学学院）；School of Economics and Management, China University of Mining and Technology（中国矿业大学经济管理学院）；DUT-BSU Joint Institute, Dalian University of Technology（大连理工大学-白俄罗斯国立大学联合学院）
UniDet-D: A Unified Dynamic Spectral Attention Model for Object Detection under Adverse Weathers
School of Interdisciplinary Science, Beijing Institute of Technology（交叉学科学院，北京理工大学）；North China University of Technology（华北理工大学）；National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology（空间智能信息处理国家重点实验室，北京理工大学）；School of Computer Science and Engineering, Nanyang Technological University（计算机科学与工程学院，南洋理工大学）
Though deep learning-based object detection methods have achieved promising results on the conventional datasets, it is still challenging to locate objects from the low-quality images captured in adverse weather conditions. The existing methods either have difficulties in balancing the tasks of image enhancement and object detection, or often ignore the latent information beneficial for detection. To alleviate this problem, we propose a novel Image-Adaptive YOLO (IA-YOLO) framework, where each image can be adaptively enhanced for better detection performance. Specifically, a differentiable ima…
Oxford Brookes University（牛津布鲁克斯大学）；University of Oxford（牛津大学）
In an autonomous driving system, perception - identification of features and objects from the environment - is crucial. In autonomous racing, high speeds and small margins demand rapid and accurate detection systems. During the race, the weather can change abruptly, causing significant degradation in perception, resulting in ineffective manoeuvres. In order to improve detection in adverse weather, deep-learning-based models typically require extensive datasets captured in such conditions - the collection of which is a tedious, laborious, and costly process. However, recent developments in Cycl…
Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Cambridge（剑桥大学）；ETH Zürich（苏黎世联邦理工学院）；The Hong Kong University of Science and Technology（香港科技大学）；The Hong Kong Polytechnic University（香港理工大学）
Reliable point cloud data is essential for perception tasks \textit{e.g.} in robotics and autonomous driving applications. Adverse weather causes a specific type of noise to light detection and ranging (LiDAR) sensor data, which degrades the quality of the point clouds significantly. To address this issue, this letter presents a novel point cloud adverse weather denoising deep learning algorithm (4DenoiseNet). Our algorithm takes advantage of the time dimension unlike deep learning adverse weather denoising methods in the literature. It performs about 10\% better in terms of intersection over …
