Recently, as many studies of autonomous vehicles have been achieved for levels 4 and 5, there has been also increasing interest in the advancement of perception, decision, and control technologies, which are the three major aspects of autonomous vehicles. As for the perception technologies achieving reliable maneuvering of autonomous vehicles, object detection by using diverse sensors (e.g., LiDAR, radar, and camera) should be prioritized. These sensors require to detect objects accurately and quickly in diverse weather conditions, but they tend to have challenges to consistently detect objects in bad weather conditions with rain, snow, or fog. Thus, in this study, based on the experimentally obtained raindrop data from precipitation conditions, we constructed a novel dataset that could test diverse network model in various precipitation conditions through the CARLA simulator. Consequently, based on our novel dataset, YOLO series, a one-stage-detector, was used to quantitatively verify how much object detection performance could be decreased under various precipitation conditions from normal to extreme heavy rain situations.
Human-in-the-loop Reasoning For Traffic Sign Detection: Collaborative Approach Yolo With Video-llava
Department of Computer Science Univ. of Applied Science Hamm-Lippstadt（计算机科学系应用科学大学哈姆-利普施塔特）；Carl von Ossietzky University of Oldenburg（卡尔·冯·奥西埃茨基旧恩堡大学）
Rain fills the atmosphere with water particles, which breaks the common assumption that light travels unaltered from the scene to the camera. While it is well-known that rain affects computer vision algorithms, quantifying its impact is difficult. In this context, we present a rain rendering pipeline that enables the systematic evaluation of common computer vision algorithms to controlled amounts of rain. We present three different ways to add synthetic rain to existing images datasets: completely physic-based; completely data-driven; and a combination of both. The physic-based rain augmentati…
To improve the robustness to rain, we present a physically-based rain rendering pipeline for realistically inserting rain into clear weather images. Our rendering relies on a physical particle simulator, an estimation of the scene lighting and an accurate rain photometric modeling to augment images with arbitrary amount of realistic rain or fog. We validate our rendering with a user study, proving our rain is judged 40% more realistic that state-of-the-art. Using our generated weather augmented Kitti and Cityscapes dataset, we conduct a thorough evaluation of deep object detection and semantic…
State-of-the-art object detection models are frequently trained offline using available datasets, such as ImageNet: large and overly diverse data that are unbalanced and hard to cluster semantically. This kind of training drops the object detection performance should the change in illumination, in the environmental conditions (e.g., rain), or in the lens positioning (out-of-focus blur) occur. We propose a decentralized hierarchical multi-agent deep reinforcement learning approach for intelligently controlling the camera and the lens focusing settings, leading to a significant improvement beyon…
Brown University（布朗大学）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）
In realistic applications of object search, robots will need to locate target objects in complex environments while coping with unreliable sensors, especially for small or hard-to-detect objects. In such settings, correlational information can be valuable for planning efficiently. Previous approaches that consider correlational information typically resort to ad-hoc, greedy search strategies. We introduce the Correlational Object Search POMDP (COS-POMDP), which models correlations while preserving optimal solutions with a reduced state space. We propose a hierarchical planning algorithm to sca…
Depth Over RGB: Automatic Evaluation of Open Surgery Skills Using Depth Camera
Technion – Israel Institute of Technology（以色列理工学院）；University of Wisconsin-Madison School of Medicine and Public Health（威斯康星大学麦迪逊分校医学与公共卫生学院）；University of Wisconsin Hospitals and Clinics（威斯康星大学医院与诊所）
Sequence-Preserving Dual-FoV Defense for Traffic Sign and Light Recognition in Autonomous Vehicles
Department of Computer Science Texas A\&M University-Corpus Christi Corpus Christi, USA；Marine System Science Texas A\&M University-Corpus Christi Corpus Christi, USA；School of Engineering \& Computing Christopher Newport University Virginia, USA
Valeo Egypt AI Research（法雷奥埃及人工智能研究院）；Cairo University（开罗大学）
Data scarcity is a bottleneck to machine learning-based perception modules, usually tackled by augmenting real data with synthetic data from simulators. Realistic models of the vehicle perception sensors are hard to formulate in closed form, and at the same time, they require the existence of paired data to be learned. In this work, we propose two unsupervised neural sensor models based on unpaired domain translations with CycleGANs and Neural Style Transfer techniques. We employ CARLA as the simulation environment to obtain simulated LiDAR point clouds, together with their annotations for dat…
Embodied Active Learning under Limited Annotation and Navigation Budget for Object Detection
LAAS-CNRS, Universite de Toulouse, CNRS（法国国家科学研究中心图卢兹分部LAAS、图卢兹大学、法国国家科学研究中心）；Department of Electrical Engineering and Automation at Aalto University（阿尔托大学电气工程与自动化系）
Object detection in adverse weather conditions for autonomous vehicles using Instruct Pix2Pix
High-Performance Embedded Systems (HPES) Lab（高性能嵌入式系统实验室）；Barcelona Supercomputing Center（巴塞罗那超级计算中心）；Computer Science Dept.（计算机科学系）；Univ. Politècnica de Catalunya（巴塞罗那理工大学）；IDEAI-UPC- Research Center（IDEAI-UPC研究中心）
Fourier Domain Adaptation for Traffic Light Detection in Adverse Weather
Manipal Institute of Technology（曼尼帕尔理工学院）；Manipal Academy of Higher Education（曼尼帕尔高等教育学院）；Wells Fargo International Solutions Private Limited（富国国际解决方案私人有限公司）
CADENet: Condition-Adaptive Asynchronous Dual-Stream Enhancement Network for Adverse Weather Perception in Autonomous Driving
Computer Science & Engineering Department, German University in Cairo (GUC), Egypt（德国开罗大学（GUC）计算机科学与工程系，埃及）；C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems, Cairo, Egypt（认知驾驶系统实验室（C-DRiVeS Lab），开罗，埃及）
An Efficient and Generalizable Transfer Learning Method for Weather Condition Detection on Ground Terminals
Center for Computational Mathematics, Faculty of Mathematics, University of Waterloo, Canada（计算数学中心，数学学院，滑铁卢大学，加拿大）；Department of Electrical and Computer Engineering, University of Manitoba, Canada（电气与计算机工程系，曼尼托巴大学，加拿大）
Beyond Clear Skies: Synthetic Seasonal and Weather Variations for Real-World Drone Detection
Institute for the Protection of Terrestrial Infrastructures, German Aerospace Center (DLR)（德国航空航天中心陆地基础设施保护研究所）；ACIDA Lab, Technical University of Applied Sciences Würzburg-Schweinfurt（维尔茨堡-施韦因富特应用技术大学ACIDA实验室）
Urban Traffic Surveillance (UTS) is a surveillance system based on a monocular and calibrated video camera that detects vehicles in an urban traffic scenario with dense traffic on multiple lanes and vehicles performing sharp turning maneuvers. UTS then tracks the vehicles using a 3D bounding box representation and a physically reasonable 3D motion model relying on an unscented Kalman filter based approach. Since UTS recovers positions, shape and motion information in a three-dimensional world coordinate system, it can be employed to recognize diverse traffic violations or to supply intelligent…
Advancing Autonomous Vehicle Intelligence: Deep Learning and Multimodal LLM for Traffic Sign Recognition and Robust Lane Detection
School of Computer Science and Engineering, Beihang University（北京航空航天大学计算机科学与工程学院）；School of Vehicle and Mobility, Tsinghua University（清华大学车辆与 Mobility 学院）
Democritus University of Thrace（色雷斯德谟克利特大学）；Information Technologies Institute, The Centre for Research & Technology, Hellas（信息技术研究所，研究与技术中心，希腊）
This paper tackles the problem of positioning a swarm of UAVs inside a completely unknown terrain, having as objective to maximize the overall situational awareness. The situational awareness is expressed by the number and quality of unique objects of interest, inside the UAVs' fields of view. YOLOv3 and a system to identify duplicate objects of interest were employed to assign a single score to each UAVs' configuration. Then, a novel navigation algorithm, capable of optimizing the previously defined score, without taking into consideration the dynamics of either UAVs or environment, is propos…
