DinoRADE: Full Spectral Radar-Camera Fusion with Vision Foundation Model Features for Multi-class Object Detection in Adverse Weather
Infineon Technologies AG, Austria（英飞凌科技公司，奥地利）；Graz University of Technology, Austria（格拉茨技术大学，奥地利）；Virtual Vehicle Research GmbH, Austria（虚拟车辆研究有限公司，奥地利）
Christof Leitgeb, Thomas Puchleitner, Max Peter Ronecker, Daniel Watzenig
备注：
Accepted to IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2026
Reliable and weather-robust perception systems are essential for safe autonomous driving and typically employ multi-modal sensor configurations to achieve comprehensive environmental awareness. While recent automotive FMCW Radar-based approaches achieved remarkable performance on detection tasks in adverse weather conditions, they exhibited limitations in resolving fine-grained spatial details particularly critical for detecting smaller and vulnerable road users (VRUs). Furthermore, existing research has not adequately addressed VRU detection in adverse weather datasets such as K-Radar. We present DinoRADE, a Radar-centered detection pipeline that processes dense Radar tensors and aggregates vision features around transformed reference points in the camera perspective via deformable cross-attention. Vision features are provided by a DINOv3 Vision Foundation Model. We present a comprehensive performance evaluation on the K-Radar dataset in all weather conditions and are among the first to report detection performance individually for five object classes. Additionally, we compare our method with existing single-class detection approaches and outperform recent Radar-camera approaches by 12.1%. The code is available under https://github.com/chr-is-tof/RADE-Net.
RADE-Net: Robust Attention Network for Radar-Only Object Detection in Adverse Weather
Infineon Technologies AG（英飞凌科技有限公司）；Institute of Visual Computing（视觉计算研究所）；Graz University of Technology（格拉茨工业大学）；Virtual Vehicle Research GmbH（虚拟车辆研究有限公司）
Gwangju Institute of Science and Technology（光州科学技术院）
Autonomous vehicles are conceived to provide safe and secure services by validating the safety standards as indicated by SOTIF-ISO/PAS-21448 (Safety of the intended functionality). Keeping in this context, the perception of the environment plays an instrumental role in conjunction with localization, planning and control modules. As a pivotal algorithm in the perception stack, object detection provides extensive insights into the autonomous vehicle's surroundings. Camera and Lidar are extensively utilized for object detection among different sensor modalities, but these exteroceptive sensors ha…
Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；College of Computer and Data Science, Fuzhou University（福州大学计算机与数据科学学院）；Institute of Information Engineering, CAS（中国科学院信息工程研究所）
The fusion of multimodal sensor streams, such as camera, lidar, and radar measurements, plays a critical role in object detection for autonomous vehicles, which base their decision making on these inputs. While existing methods exploit redundant information in good environmental conditions, they fail in adverse weather where the sensory streams can be asymmetrically distorted. These rare "edge-case" scenarios are not represented in available datasets, and existing fusion architectures are not designed to handle them. To address this challenge we present a novel multimodal dataset acquired in o…
Multi-view radar-camera fused 3D object detection provides a farther detection range and more helpful features for autonomous driving, especially under adverse weather. The current radar-camera fusion methods deliver kinds of designs to fuse radar information with camera data. However, these fusion approaches usually adopt the straightforward concatenation operation between multi-modal features, which ignores the semantic alignment with radar features and sufficient correlations across modals. In this paper, we present MVFusion, a novel Multi-View radar-camera Fusion method to achieve semantic…
LTCI, Télécom Paris, Institut Polytechnique de Paris（巴黎综合理工学院电信学院LTCI实验室）
Understanding the scene around the ego-vehicle is key to assisted and autonomous driving. Nowadays, this is mostly conducted using cameras and laser scanners, despite their reduced performances in adverse weather conditions. Automotive radars are low-cost active sensors that measure properties of surrounding objects, including their relative speed, and have the key advantage of not being impacted by rain, snow or fog. However, they are seldom used for scene understanding due to the size and complexity of radar raw data and the lack of annotated datasets. Fortunately, recent open-sourced datase…
4D radar is recognized for its resilience and cost-effectiveness under adverse weather conditions, thus playing a pivotal role in autonomous driving. While cameras and LiDAR are typically the primary sensors used in perception modules for autonomous vehicles, radar serves as a valuable supplementary sensor. Unlike LiDAR and cameras, radar remains unimpaired by harsh weather conditions, thereby offering a dependable alternative in challenging environments. Developing radar-based 3D object detection not only augments the competency of autonomous vehicles but also provides economic benefits. In r…
SemRaFiner: Panoptic Segmentation in Sparse and Noisy Radar Point Clouds
CARIAD SE；University of Bonn（波恩大学）；Technical University of Munich（慕尼黑工业大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉马尔机器学习与人工智能研究所）
Autonomous driving requires a detailed understanding of complex driving scenes. The redundancy and complementarity of the vehicle's sensors provide an accurate and robust comprehension of the environment, thereby increasing the level of performance and safety. This thesis focuses the on automotive RADAR, which is a low-cost active sensor measuring properties of surrounding objects, including their relative speed, and has the key advantage of not being impacted by adverse weather conditions. With the rapid progress of deep learning and the availability of public driving datasets, the perception…
Autonomous radar has been an integral part of advanced driver assistance systems due to its robustness to adverse weather and various lighting conditions. Conventional automotive radars use digital signal processing (DSP) algorithms to process raw data into sparse radar pins that do not provide information regarding the size and orientation of the objects. In this paper, we propose a deep-learning based algorithm for radar object detection. The algorithm takes in radar data in its raw tensor representation and places probabilistic oriented bounding boxes around the detected objects in bird's-e…
WS-DETR: Robust Water Surface Object Detection through Vision-Radar Fusion with Detection Transformer
College of Electronics and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Graz University of Technology（格拉茨工业大学）；Virtual Vehicle Research（虚拟车辆研究院）
University of Oxford（牛津大学）；Oxford Robotics Institute（牛津机器人研究所）
This paper presents an efficient annotation procedure and an application thereof to end-to-end, rich semantic segmentation of the sensed environment using FMCW scanning radar. We advocate radar over the traditional sensors used for this task as it operates at longer ranges and is substantially more robust to adverse weather and illumination conditions. We avoid laborious manual labelling by exploiting the largest radar-focused urban autonomy dataset collected to date, correlating radar scans with RGB cameras and LiDAR sensors, for which semantic segmentation is an already consolidated procedur…
Leveraging Self-Supervised Instance Contrastive Learning for Radar Object Detection
Artificial and Natural Intelligence Toulouse Institute（图卢兹人工与自然智能研究所）；Université de Toulouse（图卢兹大学）；ISAE-SUPAERO（法国高等航空航天学院）；CerCO（认知与认知科学研究中心）；NXP Semiconductors（恩智浦半导体）
CARIAD SE；University of Bonn（波恩大学）；University of Oxford（牛津大学）；Lamarr Institute for Machine Learning and Artificial Intelligence（拉玛尔机器学习与人工智能研究所）
Scene understanding is crucial for autonomous robots in dynamic environments for making future state predictions, avoiding collisions, and path planning. Camera and LiDAR perception made tremendous progress in recent years, but face limitations under adverse weather conditions. To leverage the full potential of multi-modal sensor suites, radar sensors are essential for safety critical tasks and are already installed in most new vehicles today. In this paper, we address the problem of semantic segmentation of moving objects in radar point clouds to enhance the perception of the environment with…
Recent works have shown the superior robustness of four-dimensional (4D) Radar-based three-dimensional (3D) object detection in adverse weather conditions. However, processing 4D Radar data remains a challenge due to the large data size, which require substantial amount of memory for computing and storage. In previous work, an online density reduction is performed on the 4D Radar Tensor (4DRT) to reduce the data size, in which the density reduction level is chosen arbitrarily. However, the impact of density reduction on the detection performance and memory consumption remains largely unknown. …
mRadNet: A Compact Radar Object Detector with MetaFormer
School of Electrical Engineering and Computer Science, University of Ottawa, Canada（渥太华大学电气与计算机工程学院，加拿大）；tsensor Cortek Inc., Canada（加拿大tsensor Cortek公司）
We propose DeepFusion, a modular multi-modal architecture to fuse lidars, cameras and radars in different combinations for 3D object detection. Specialized feature extractors take advantage of each modality and can be exchanged easily, making the approach simple and flexible. Extracted features are transformed into bird's-eye-view as a common representation for fusion. Spatial and semantic alignment is performed prior to fusing modalities in the feature space. Finally, a detection head exploits rich multi-modal features for improved 3D detection performance. Experimental results for lidar-came…
Ocean University of China（中国海洋大学）；Meteorological Bureau of Tianjin Municipality（天津市气象局）；Mississippi State University（密西西比州立大学）
Synthetic aperture radar (SAR) image change detection is a vital yet challenging task in the field of remote sensing image analysis. Most previous works adopt a self-supervised method which uses pseudo-labeled samples to guide subsequent training and testing. However, deep networks commonly require many high-quality samples for parameter optimization. The noise in pseudo-labels inevitably affects the final change detection performance. To solve the problem, we propose a Graph-based Knowledge Supplement Network (GKSNet). To be more specific, we extract discriminative information from the existi…
Learning Structurally Consistent Representations for Multi-View Radar Semantic Segmentation
School of Computing, Engineering and Mathematical Sciences, La Trobe University（拉筹伯大学计算、工程与数学科学学院）；La Trobe Institute for Sustainable Agriculture & Food (LISAF)（拉筹伯大学可持续农业与食品研究所）；School of Engineering and Applied Sciences, GIFT University（GIFT大学工程与应用科学学院）；Commonwealth Scientific and Industrial Research Organisation (CSIRO)（澳大利亚联邦科学与工业研究组织）
University of Washington（华盛顿大学）；Silkwave Holdings Limited（Silkwave控股有限公司）
Various autonomous or assisted driving strategies have been facilitated through the accurate and reliable perception of the environment around a vehicle. Among the commonly used sensors, radar has usually been considered as a robust and cost-effective solution even in adverse driving scenarios, e.g., weak/strong lighting or bad weather. Instead of considering to fuse the unreliable information from all available sensors, perception from pure radar data becomes a valuable alternative that is worth exploring. In this paper, we propose a deep radar object detection network, named RODNet, which is…
WaterScenes: A Multi-Task 4D Radar-Camera Fusion Dataset and Benchmarks for Autonomous Driving on Water Surfaces
University of Liverpool（利物浦大学）；Wuhan University of Technology（武汉理工大学）；Nantong University（南通大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
Wavelet-based Multi-View Fusion of 4D Radar Tensor and Camera for Robust 3D Object Detection
Thrust of Artificial Intelligence, Hong Kong University of Science and Technology (Guangzhou)（人工智能 thrust，香港科技大学（广州））；Momoniai AI；Department of Mechanical Engineering, Massachusetts Institute of Technology（机械工程系，麻省理工学院）；School of Information Engineering, Yancheng Institute of Technology（信息工程学院，盐城科技学院）；College of Information Science and Electronic Engineering, Zhejiang University（信息科学与电子工程学院，浙江大学）；Institute for Math & AI, Wuhan University（数学与人工智能研究所，武汉大学）；College of Science and Engineering and the Centre for AI and Data Science Innovation, James Cook University（科学与工程学院及人工智能与数据科学创新中心，詹姆斯库克大学）；School of Transportation, Southeast University（交通运输学院，东南大学）
Autonomous perception requires high-quality environment sensing in the form of 3D bounding boxes of dynamic objects. The primary sensors used in automotive systems are light-based cameras and LiDARs. However, they are known to fail in adverse weather conditions. Radars can potentially solve this problem as they are barely affected by adverse weather conditions. However, specular reflections of wireless signals cause poor performance of radar point clouds. We introduce Pointillism, a system that combines data from multiple spatially separated radars with an optimal separation to mitigate these …
