RC-BEVFusion: A Plug-In Module for Radar-Camera Bird's Eye View Feature Fusion
Stellantis, Opel Automobile GmbH（斯泰兰蒂斯集团欧宝汽车有限公司）；Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau（莱茵兰-普法尔茨凯泽斯劳滕-兰道工业大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
Lukas Stäcker, Shashank Mishra, Philipp Heidenreich, Jason Rambach, Didier Stricker
Radars and cameras belong to the most frequently used sensors for advanced driver assistance systems and automated driving research. However, there has been surprisingly little research on radar-camera fusion with neural networks. One of the reasons is a lack of large-scale automotive datasets with radar and unmasked camera data, with the exception of the nuScenes dataset. Another reason is the difficulty of effectively fusing the sparse radar point cloud on the bird's eye view (BEV) plane with the dense images on the perspective plane. The recent trend of camera-based 3D object detection using BEV features has enabled a new type of fusion, which is better suited for radars. In this work, we present RC-BEVFusion, a modular radar-camera fusion network on the BEV plane. We propose BEVFeatureNet, a novel radar encoder branch, and show that it can be incorporated into several state-of-the-art camera-based architectures. We show significant performance gains of up to 28% increase in the nuScenes detection score, which is an important step in radar-camera fusion research. Without tuning our model for the nuScenes benchmark, we achieve the best result among all published methods in the radar-camera fusion category.
Integrating LiDAR and Camera information into Bird's-Eye-View (BEV) has become an essential topic for 3D object detection in autonomous driving. Existing methods mostly adopt an independent dual-branch framework to generate LiDAR and camera BEV, then perform an adaptive modality fusion. Since point clouds provide more accurate localization and geometry information, they could serve as a reliable spatial prior to acquiring relevant semantic information from the images. Therefore, we design a LiDAR-Guided View Transformer (LGVT) to effectively obtain the camera representation in BEV space and th…
BEVSpread: Spread Voxel Pooling for Bird's-Eye-View Representation in Vision-based Roadside 3D Object Detection
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Polytechnic Institute, Zhejiang University Baidu（浙江大学百度理工学院）；Zhejiang – Singapore Innovation and AI Joint Research Lab（浙新创新与人工智能联合研究实验室）
Semantics-Guided Multimodal Masked Autoencoder Pretraining for 3D BEV Object Detection
University of Lincoln, Lincoln Centre for Autonomous Systems（林肯大学，林肯自主系统中心）；University of Cambridge, Institute for Manufacturing, Department of Engineering（剑桥大学，制造研究所，工程系）
Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection
Dept. of Electronic and Computer Engineering and the Data Driven Computer Engineering Research Centre, University of Limerick（电子与计算机工程系和数据驱动计算机工程研究中心，利默里克大学）；Valeo Vision Systems（瓦莱欧视觉系统）；Queen Mary University of London（伦敦女王大学）
Safety-Oriented Trajectory Ranking from BEV Perception Outputs
University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Research Institute CODE, University of the Bundeswehr Munich（德国联邦国防军慕尼黑大学CODE研究院）
UniBEVFusion: Unified Radar-Vision BEVFusion for 3D Object Detection
Institute of Deep Perception Technology, JITRI（捷瑞深度感知技术研究所）；University of Liverpool（利物浦大学）；Xi’an Jiaotong-Liverpool University（西交利物浦大学）；XJTLU-JITRI Academy of Technology, Xi’an Jiaotong-Liverpool University（西交利物浦大学-捷瑞技术学院）；The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
SafeMap: Robust HD Map Construction from Incomplete Observations
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；School of Computer Science, Hefei University of Technology（合肥工业大学计算机学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；School of Computer Science, Wuhan University（武汉大学计算机学院）；National University of Singapore（新加坡国立大学）；Intelligent Interconnected Systems Laboratory of Anhui Province (Hefei University of Technology)（安徽省智能互联系统实验室（合肥工业大学））
South China University of Technology（华南理工大学）；DexForce Technology Co., Ltd.（德方科技有限公司）；Magic Leap（魔力跃动）；Peng Cheng Laboratory（鹏城实验室）
Detecting objects from LiDAR point clouds is of tremendous significance in autonomous driving. In spite of good progress, accurate and reliable 3D detection is yet to be achieved due to the sparsity and irregularity of LiDAR point clouds. Among existing strategies, multi-view methods have shown great promise by leveraging the more comprehensive information from both bird's eye view (BEV) and range view (RV). These multi-view methods either refine the proposals predicted from single view via fused features, or fuse the features without considering the global spatial context; their performance i…
Can BEV Perception Gracefully Degrade under Sensor Failures?
Tianjin Key Laboratory of Intelligent Unmanned Swarm Technology and System（天津智能无人群技术与系统重点实验室）；School of Electrical and Information Engineering, Tianjin University（天津大学电气与信息工程学院）；Key Laboratory of System Control and Information Processing, Ministry of Education of China（系统控制与信息处理重点实验室，中华人民共和国教育部）
M-BEV: Masked BEV Perception for Robust Autonomous Driving
Shenzhen Institute of Advanced Technology, Chinese Academy of Science（中国科学院深圳先进技术研究院）；University of Chinese Academy of Science（中国科学院大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）
MapFusion: A Novel BEV Feature Fusion Network for Multi-modal Map Construction
Beijing Academy of Artificial Intelligence（北京人工智能研究院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Pennsylvania State University（宾夕法尼亚州立大学）；Department of Biomedical Engineering, Hefei University of Technology（合肥工业大学生物医学工程系）
Visual classification can be divided into coarse-grained and fine-grained classification. Coarse-grained classification represents categories with a large degree of dissimilarity, such as the classification of cats and dogs, while fine-grained classification represents classifications with a large degree of similarity, such as cat species, bird species, and the makes or models of vehicles. Unlike coarse-grained visual classification, fine-grained visual classification often requires professional experts to label data, which makes data more expensive. To meet this challenge, many approaches pro…
ContrastAlign: Toward Robust BEV Feature Alignment via Contrastive Learning for Multi-Modal 3D Object Detection
School of Computer Science & Technology, Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University（计算机科学与技术学院，北京交通大数据挖掘与具身智能重点实验室，北京交通大学）；Horizon Robotics；Nanyang Technological University（南洋理工大学）；University of Macau（澳门大学）；School of Information Science and Engineering, Yanshan University（信息科学与工程学院，燕山大学）；School of Computer Science and Technology, Harbin Institute of Technology（计算机科学与技术学院，哈尔滨理工大学）；School of Information Technology and Electrical Engineering, The University of Queensland（信息技术与电气工程学院，昆士兰大学）
Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Prediction
SenseTime Research（商汤科技研究院）；The University of Hong Kong（香港大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；The Chinese University of Hong Kong（香港中文大学）
