Accurate moving object segmentation is an essential task for autonomous driving. It can provide effective information for many downstream tasks, such as collision avoidance, path planning, and static map construction. How to effectively exploit the spatial-temporal information is a critical question for 3D LiDAR moving object segmentation (LiDAR-MOS). In this work, we propose a novel deep neural network exploiting both spatial-temporal information and different representation modalities of LiDAR scans to improve LiDAR-MOS performance. Specifically, we first use a range image-based dual-branch structure to separately deal with spatial and temporal information that can be obtained from sequential LiDAR scans, and later combine them using motion-guided attention modules. We also use a point refinement module via 3D sparse convolution to fuse the information from both LiDAR range image and point cloud representations and reduce the artifacts on the borders of the objects. We verify the effectiveness of our proposed approach on the LiDAR-MOS benchmark of SemanticKITTI. Our method outperforms the state-of-the-art methods significantly in terms of LiDAR-MOS IoU. Benefiting from the devised coarse-to-fine architecture, our method operates online at sensor frame rate. The implementation of our method is available as open source at: https://github.com/haomo-ai/MotionSeg3D.
Doppler velocity-based algorithm for Clustering and Velocity Estimation of moving objects
Zhejiang University(浙江大学)
针对FMCW激光雷达的特点,提出了一种基于多普勒速度的聚类和速度估计算法,实现了高精度、单扫描、实时的运动状态检测和速度估计。我们证明了同一物体上多普勒速度的连续性。基于这一原理,我们通过区域生长聚类算法实现了运动目标和静止背景的区分。利用所得到的静止背景,采用最小二乘法对FMCW激光雷达的速度进行估计。然后,我们估计的LiDAR速度和多普勒速度的运动物体通过聚类得到的运动物体的速度。为了确保实时处理,我们设置了适当的最小二乘参数。同时,为了验证算法的有效性,在自动驾驶仿真平台CARLA上建立了FMCW激光雷达模型,进行数据的生成。实验结果表明,该算法在Ryzen 3600 x CPU的运算能力下,每秒至少可以处理450万个点,估计150个运动物体的速度,运动状态检测精度超过99%,速度估计精度为0.1 m/s。
BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework
Wangxuan Institute of Computer Technology, Peking University(北京大学王选计算机研究所);DAMO Academy, Alibaba Group(阿里巴巴集团达摩院);Shenzhen Campus of Sun Yat-sen University(中山大学深圳校区)
Improving Robustness of LiDAR-Camera Fusion Model against Weather Corruption from Fusion Strategy Perspective
Nanyang Technological University(南洋理工大学);East China Normal University(华东师范大学);Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);New York University(纽约大学)
LiDAR-BIND-T: Improved and Temporally Consistent Sensor Modality Translation and Fusion for Robotic Applications
IDLab - Faculty of Applied Engineering, University of Antwerp - imec(IDLab - 应用工程学院,安特卫普大学 - imec);Cosys-Lab - Faculty of Applied Engineering, University of Antwerp(Cosys-Lab - 应用工程学院,安特卫普大学);Flanders Make Strategic Research Centre(弗拉芒制造战略研究中心)
本文提出 LiDAR-BIND-T,通过时间嵌入相似度、运动对齐损失和窗口化时间融合增强异构传感器到 LiDAR 的转换,并以新指标验证其提升 SLAM 稳定性与精度。
CLLAP: Contrastive Learning-based LiDAR-Augmented Pretraining for Enhanced Radar-Camera Fusion
Wuhan University of Technology(武汉理工大学);University of North Carolina at Charlotte(北卡罗来纳州立大学);Wuhan University(武汉大学);Southwest Jiaotong University(西南交通大学);The Hong Kong Polytechnic University(香港理工大学)
FlatFusion: Delving into Details of Sparse Transformer-based Camera-LiDAR Fusion for Autonomous Driving
School of Computer Science and School of Artificial Intelligence, Shanghai Jiao Tong University(计算机科学学院和人工智能学院,上海交通大学);Carnegie Mellon University(卡内基梅隆大学)
MS-Occ: Multi-Stage LiDAR-Camera Fusion for 3D Semantic Occupancy Prediction
School of Optical-Electrical and Computer Engineering, University of Shanghai for Science and Technology(光学电子与计算机工程学院,上海科学技术大学);School of Automotive Studies, Tongji University(汽车学院,同济大学);Momoni AI;College of Science and Engineering, James Cook University(科学与工程学院,詹姆斯库克大学);School of Engineering, Swinburne University of Technology(工程学院,斯威本技术大学);School of Electrical and Electronic Engineering, Nanyang Technological University(电气与电子工程学院,南洋理工大学)
Joint Multi-Object Detection and Tracking with Camera-LiDAR Fusion for Autonomous Driving
Southern University of Science and Technology(南方科技大学);Sifakis Research Institute of Trustworthy Autonomous Systems, Southern University of Science and Technology(南方科技大学西法基斯可信自主系统研究院);Pazhou Lab(琶洲实验室)
Timely Fusion of Surround Radar/Lidar for Object Detection in Autonomous Driving Systems
City University of Hong Kong(香港城市大学);Yale University(耶鲁大学);The Chinese University of Hong Kong(香港中文大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
LFP: Efficient and Accurate End-to-End Lane-Level Planning via Camera-LiDAR Fusion
School of Computer Science and Technology, University of Science and Technology of China (USTC)(中国科学技术大学计算机科学与技术学院);SenseTime Research(商汤科技研究院);Shanghai AI Laboratory(上海人工智能实验室);School of Artificial Intelligence and Data Science, University of Science and Technology of China (USTC)(中国科学技术大学人工智能与数据科学学院)
LiDAR-VGGT: Cross-Modal Coarse-to-Fine Fusion for Globally Consistent and Metric-Scale Dense Mapping
State Key Laboratory of Industrial Control Technology, Institute of Cyber-Systems and Control, Zhejiang University(工业控制技术国家重点实验室,系统与控制研究院,浙江大学);Differential Robot Technology Co., Ltd.(差分机器人技术有限公司);College of Intelligence Science and Technology, National University of Defense Technology(智能科学与技术学院,国防科技大学)
HeLiMOS: A Dataset for Moving Object Segmentation in 3D Point Clouds From Heterogeneous LiDAR Sensors
School of Electrical Engineering, KAIST (Korea Advanced Institute of Science and Technology)(韩国科学技术院电气工程学院);Center for Robotics, University of Bonn(波恩大学机器人中心);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究所)