Depth estimation is critical in autonomous driving for interpreting 3D scenes accurately. Recently, radar-camera depth estimation has become of sufficient interest due to the robustness and low-cost properties of radar. Thus, this paper introduces a two-stage, end-to-end trainable Confidence-aware Fusion Net (CaFNet) for dense depth estimation, combining RGB imagery with sparse and noisy radar point cloud data. The first stage addresses radar-specific challenges, such as ambiguous elevation and noisy measurements, by predicting a radar confidence map and a preliminary coarse depth map. A novel approach is presented for generating the ground truth for the confidence map, which involves associating each radar point with its corresponding object to identify potential projection surfaces. These maps, together with the initial radar input, are processed by a second encoder. For the final depth estimation, we innovate a confidence-aware gated fusion mechanism to integrate radar and image features effectively, thereby enhancing the reliability of the depth map by filtering out radar noise. Our methodology, evaluated on the nuScenes dataset, demonstrates superior performance, improving upon the current leading model by 3.2% in Mean Absolute Error (MAE) and 2.7% in Root Mean Square Error (RMSE). Code: https://github.com/harborsarah/CaFNet
Dense depth estimation plays a key role in multiple applications such as robotics, 3D reconstruction, and augmented reality. While sparse signal, e.g., LiDAR and Radar, has been leveraged as guidance for enhancing dense depth estimation, the improvement is limited due to its low density and imbalanced distribution. To maximize the utility from the sparse source, we propose $S^3$ technique, which expands the depth value from sparse cues while estimating the confidence of expanded region. The proposed $S^3$ can be applied to various guided depth estimation approaches and trained end-to-end at di…
National Yang Ming Chiao Tung University（阳明交通大学）；University of Washington（华盛顿大学）
This paper introduces a novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR), that includes synchronized vision and radio signal components. This dataset is created using cross-calibrated mmWave radar sensors and a monocular RGB camera for cross-modality training of radar-based human pose estimation. There are two advantages of using mmWave radar to perform human pose estimation. First, it is robust to dark and low-light conditions. Second, it is not visually perceivable by humans and thus, can be widely applied to applications with privacy concerns, e.g., survei…
University of Washington（华盛顿大学）；Silkwave Holdings Limited（Silkwave控股有限公司）
Radar is usually more robust than the camera in severe driving scenarios, e.g., weak/strong lighting and bad weather. However, unlike RGB images captured by a camera, the semantic information from the radar signals is noticeably difficult to extract. In this paper, we propose a deep radar object detection network (RODNet), to effectively detect objects purely from the carefully processed radar frequency data in the format of range-azimuth frequency heatmaps (RAMaps). Three different 3D autoencoder based architectures are introduced to predict object confidence distribution from each snippet of…
calibfusion: Transformer-Based Differentiable Calibration for Radar-Camera Fusion Detection in Water-Surface Environments
School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）
Sparse4D-Radar: An Efficient and Robust Framework for Surround-View 3D Object Detection via 4D Radar-Camera Fusion
State Key Laboratory of Ocean Sensing and Ocean College, Zhejiang University（浙江大学海洋传感与海洋学院海洋传感技术国家重点实验室）；Engineering Research Center of Oceanic Sensing Technology and Equipment, Ministry of Education, Zhejiang University（浙江大学海洋传感技术与装备教育部工程研究中心）；Donghai Laboratory（东海实验室）；Ocean College, Institute of Marine Electronics and Intelligent Systems, Zhejiang University（浙江大学海洋学院海洋电子与智能系统研究所）
Improving Greenland Bed Topography Mapping with Uncertainty-Aware Graph Learning on Sparse Radar Data
iHARP, University of Maryland Baltimore County (UMBC)（iHARP，马里兰大学巴尔的摩县分校）；Department of Information Systems, University of Maryland Baltimore County (UMBC)（信息系统系，马里兰大学巴尔的摩县分校）
RC-GeoCP: Geometric Consensus for 4D Radar-Camera Collaborative Perception
College of Information Science and Electronic Engineering, Zhejiang University（浙江大学信息科学与电子工程学院）；School of Automotive Studies, Tongji University（同济大学汽车学院）；Thrust of Artificial Intelligence, Hong Kong University of Science and Technology（香港科技大学人工智能研究所）
Data-Driven Uncertainty-Aware Forecasting of Sea Ice Conditions in the Gulf of Ob Based on Satellite Radar Imagery
Applied AI Center, Skolkovo Institute of Science and Technology, Moscow, Russia（斯克尔科沃科学与技术研究所应用人工智能中心）；Moscow Institute of Physics and Technology, Dolgoprudny, Russia（莫斯科物理技术学院）；Autonomous Non-Profit Organization Artificial Intelligence Research Institute (AIRI), Moscow, Russia（自主非营利组织人工智能研究 institute（AIRI））
