Semantic understanding of the surrounding environment is essential for automated vehicles. The recent publication of the SemanticKITTI dataset stimulates the research on semantic segmentation of LiDAR point clouds in urban scenarios. While most existing approaches predict sparse pointwise semantic classes for the sparse input LiDAR scan, we propose PillarSegNet to be able to output a dense semantic grid map. In contrast to a previously proposed grid map method, PillarSegNet uses PointNet to learn features directly from the 3D point cloud and then conducts 2D semantic segmentation in the top view. To train and evaluate our approach, we use both sparse and dense ground truth, where the dense ground truth is obtained from multiple superimposed scans. Experimental results on the SemanticKITTI dataset show that PillarSegNet achieves a performance gain of about 10% mIoU over the state-of-the-art grid map method.
We propose a simple yet effective proposal-free architecture for lidar panoptic segmentation. We jointly optimize both semantic segmentation and class-agnostic instance classification in a single network using a pillar-based bird's-eye view representation. The instance classification head learns pairwise affinity between pillars to determine whether the pillars belong to the same instance or not. We further propose a local clustering algorithm to propagate instance ids by merging semantic segmentation and affinity predictions. Our experiments on nuScenes dataset show that our approach outperfo…
Recent works recognized lidars as an inherently streaming data source and showed that the end-to-end latency of lidar perception models can be reduced significantly by operating on wedge-shaped point cloud sectors rather then the full point cloud. However, due to use of cartesian coordinate systems these methods represent the sectors as rectangular regions, wasting memory and compute. In this work we propose using a polar coordinate system and make two key improvements on this design. First, we increase the spatial context by using multi-scale padding from neighboring sectors: preceding sector…
University of California at Riverside（加州大学河滨分校）；Toyota North America R&D Labs（丰田北美研发实验室）
3D object detection plays a fundamental role in enabling autonomous driving, which is regarded as the significant key to unlocking the bottleneck of contemporary transportation systems from the perspectives of safety, mobility, and sustainability. Most of the state-of-the-art (SOTA) object detection methods from point clouds are developed based on a single onboard LiDAR, whose performance will be inevitably limited by the range and occlusion, especially in dense traffic scenarios. In this paper, we propose \textit{PillarGrid}, a novel cooperative perception method fusing information from multi…
Karlsruhe Institute of Technology（卡尔斯鲁厄理工学院）；Stellantis, Opel Automobile GmbH（斯特兰蒂斯，欧宝汽车有限公司）
At the heart of all automated driving systems is the ability to sense the surroundings, e.g., through semantic segmentation of LiDAR sequences, which experienced a remarkable progress due to the release of large datasets such as SemanticKITTI and nuScenes-LidarSeg. While most previous works focus on sparse segmentation of the LiDAR input, dense output masks provide self-driving cars with almost complete environment information. In this paper, we introduce MASS - a Multi-Attentional Semantic Segmentation model specifically built for dense top-view understanding of the driving scenes. Our framew…
Beijing Institute of Technology（北京理工大学）；Baidu Research（百度研究院）；Inception Institute of Artificial Intelligence（人工智能起源研究所）；National Engineering Laboratory of Deep Learning Technology and Application（深度学习技术及应用国家工程实验室）；University of Kentucky（肯塔基大学）
Existing LiDAR-based 3D object detectors usually focus on the single-frame detection, while ignoring the spatiotemporal information in consecutive point cloud frames. In this paper, we propose an end-to-end online 3D video object detector that operates on point cloud sequences. The proposed model comprises a spatial feature encoding component and a spatiotemporal feature aggregation component. In the former component, a novel Pillar Message Passing Network (PMPNet) is proposed to encode each discrete point cloud frame. It adaptively collects information for a pillar node from its neighbors by …
Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；University of Toronto（多伦多大学）
A fast and accurate panoptic segmentation system for LiDAR point clouds is crucial for autonomous driving vehicles to understand the surrounding objects and scenes. Existing approaches usually rely on proposals or clustering to segment foreground instances. As a result, they struggle to achieve real-time performance. In this paper, we propose a novel real-time end-to-end panoptic segmentation network for LiDAR point clouds, called CPSeg. In particular, CPSeg comprises a shared encoder, a dual-decoder, and a cluster-free instance segmentation head, which is able to dynamically pillarize foregro…
3D object detection with LiDAR point clouds plays an important role in autonomous driving perception module that requires high speed, stability and accuracy. However, the existing point-based methods are challenging to reach the speed requirements because of too many raw points, and the voxel-based methods are unable to ensure stable speed because of the 3D sparse convolution. In contrast, the 2D grid-based methods, such as PointPillar, can easily achieve a stable and efficient speed based on simple 2D convolution, but it is hard to get the competitive accuracy limited by the coarse-grained po…
Enhancing Lidar-based Object Detection in Adverse Weather using Offset Sequences in Time
Institute for Automotive Engineering RWTH Aachen University（亚琛工业大学汽车工程研究所）；Institute of Imaging and Computer Vision RWTH Aachen University（亚琛工业大学图像与计算机视觉研究所）
Multistream Network for LiDAR and Camera-based 3D Object Detection in Outdoor Scenes
Department of Computer Science, The University of Western Australia（西澳大学计算机科学系）；School of Computing & Information Systems, The University of Melbourne（墨尔本大学计算与信息系）
Opel Automobile GmbH（欧宝汽车公司）；Institute of Measurement and Control Systems, Karlsruhe Institute of Technology (KIT)（卡尔斯鲁厄理工学院测量与控制系统研究所）
3D pedestrian detection is a challenging task in automated driving because pedestrians are relatively small, frequently occluded and easily confused with narrow vertical objects. LiDAR and camera are two commonly used sensor modalities for this task, which should provide complementary information. Unexpectedly, LiDAR-only detection methods tend to outperform multisensor fusion methods in public benchmarks. Recently, PointPainting has been presented to eliminate this performance drop by effectively fusing the output of a semantic segmentation network instead of the raw image information. In thi…
Real-Aug: Realistic Scene Synthesis for LiDAR Augmentation in 3D Object Detection
Inspur Electronic Information Industry Co.,Ltd.（浪潮电子信息产业股份有限公司）；Centre for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人中心）；Institute of Automation, CAS（中国科学院自动化研究所）
PAN: Pillars-Attention-Based Network for 3D Object Detection
Department of Electronic and Computer Engineering, Lero (the Research Ireland Centre for Software), and the Data Driven Computer Engineering (D 2 iCE) Research Centre at the University of Limerick（电子与计算机工程系，Lero（爱尔兰软件研究中心），以及利默里克大学数据驱动计算机工程（D 2 iCE）研究中心）；Provizio, Future Mobility Campus Ireland, Shannon Free Zone（Provizio，Future Mobility Campus Ireland，Shannon Free Zone）
ContrastMotion: Self-supervised Scene Motion Learning for Large-Scale LiDAR Point Clouds
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；Rhino（犀牛（公司））；The Chinese University of Hong Kong（香港中文大学）；OPPO Research Institute（OPPO研究院）；Zhejiang Lab（之江实验室）；ShanghaiTech University（上海科技大学）
PillarTrack:Boosting Pillar Representation for Transformer-based 3D Single Object Tracking on Point Clouds
School of Mathematics and Computer Science, Nanchang University（南昌大学数学与计算机科学学院）；School of Automation, Southeast University（东南大学自动化学院）；Aberdeen Institute of Data Science and Artificial Intelligence, South China Normal University（华南师范大学 Aberdeen 数据科学与人工智能研究院）；Houmo AI
Multi-Correlation Siamese Transformer Network with Dense Connection for 3D Single Object Tracking
Zhengzhou University（郑州大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Nullmax Inc（Nullmax公司）
