Accurate motion understanding of the dynamic objects within the scene in bird's-eye-view (BEV) is critical to ensure a reliable obstacle avoidance system and smooth path planning for autonomous vehicles. However, this task has received relatively limited exploration when compared to object detection and segmentation with only a few recent vision-based approaches presenting preliminary findings that significantly deteriorate in low-light, nighttime, and adverse weather conditions such as rain. Conversely, LiDAR and radar sensors remain almost unaffected in these scenarios, and radar provides key velocity information of the objects. Therefore, we introduce BEVMOSNet, to our knowledge, the first end-to-end multimodal fusion leveraging cameras, LiDAR, and radar to precisely predict the moving objects in BEV. In addition, we perform a deeper analysis to find out the optimal strategy for deformable cross-attention-guided sensor fusion for cross-sensor knowledge sharing in BEV. While evaluating BEVMOSNet on the nuScenes dataset, we show an overall improvement in IoU score of 36.59% compared to the vision-based unimodal baseline BEV-MoSeg (Sigatapu et al., 2023), and 2.35% compared to the multimodel SimpleBEV (Harley et al., 2022), extended for the motion segmentation task, establishing this method as the state-of-the-art in BEV motion segmentation.
Detection of moving objects is a very important task in autonomous driving systems. After the perception phase, motion planning is typically performed in Bird's Eye View (BEV) space. This would require projection of objects detected on the image plane to top view BEV plane. Such a projection is prone to errors due to lack of depth information and noisy mapping in far away areas. CNNs can leverage the global context in the scene to project better. In this work, we explore end-to-end Moving Object Detection (MOD) on the BEV map directly using monocular images as input. To the best of our knowled…
CV-MOS: A Cross-View Model for Motion Segmentation
Xingzhi College, South China Normal University（华南师范大学行知学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学学院）；Hong Kong University of Science and Technology（香港科技大学）
BEVScope: Enhancing Self-Supervised Depth Estimation Leveraging Bird's-Eye-View in Dynamic Scenarios
Tsinghua University（清华大学）；University of Science and Technology Beijing（北京科技大学）；University of Chinese Academy of Sciences（中国科学院大学）；Southeast University（东南大学）
BEVUDA++: Geometric-aware Unsupervised Domain Adaptation for Multi-View 3D Object Detection
National Key Laboratory for Multimedia Information Processing in School of Computer Science at Peking University（北京大学计算机科学学院多媒体信息处理国家级重点实验室）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Nanjing University（南京大学）
Moving object detection and segmentation is an essential task in the Autonomous Driving pipeline. Detecting and isolating static and moving components of a vehicle's surroundings are particularly crucial in path planning and localization tasks. This paper proposes a novel real-time architecture for motion segmentation of Light Detection and Ranging (LiDAR) data. We use three successive scans of LiDAR data in 2D Bird's Eye View (BEV) representation to perform pixel-wise classification as static or moving. Furthermore, we propose a novel data augmentation technique to reduce the significant clas…
Autonomous navigation requires scene understanding of the action-space to move or anticipate events. For planner agents moving on the ground plane, such as autonomous vehicles, this translates to scene understanding in the bird's-eye view (BEV). However, the onboard cameras of autonomous cars are customarily mounted horizontally for a better view of the surrounding. In this work, we study scene understanding in the form of online estimation of semantic BEV maps using the video input from a single onboard camera. We study three key aspects of this task, image-level understanding, BEV level unde…
MV-MOS: Multi-View Feature Fusion for 3D Moving Object Segmentation
Research Institute of Trustworthy Autonomous Systems（可信自主系统研究院）；Southern University of Science and Technology（南方科技大学）；School of Electronic and Information Engineering, South China Normal University（华南师范大学电子与信息工程学院）；College of Intelligence Science and Technology, National University of Defense Technology（国防科技大学智能科学学院）
Institute of Automation, Chinese Academy of Sciences (CASIA)（中国科学院自动化研究所）；University of Chinese Academy of Sciences (UCAS)（中国科学院大学）；HKISI_CAS（香港创新研究院（中国科学院））
The transformation of features from 2D perspective space to 3D space is essential to multi-view 3D object detection. Recent approaches mainly focus on the design of view transformation, either pixel-wisely lifting perspective view features into 3D space with estimated depth or grid-wisely constructing BEV features via 3D projection, treating all pixels or grids equally. However, choosing what to transform is also important but has rarely been discussed before. The pixels of a moving car are more informative than the pixels of the sky. To fully utilize the information contained in images, the v…
KDMOS:Knowledge Distillation for Motion Segmentation
School of Electronic and Information Engineering, South China Normal University（南方科技大学电子与信息工程学院）；College of Electronics & Information Engineering, Shanghai Research Institute for Intelligent Autonomous Systems, the State Key Laboratory of Intelligent Autonomous Systems, and Frontiers Science Center for Intelligent Autonomous Systems, Tongji University（上海智能自主系统研究院、智能自主系统国家重点实验室及前沿科学中心，同济大学电子与信息工程学院）；College of Big Data and Internet, Shenzhen Technology University（深圳技术大学大数据与互联网学院）
