Background modeling is widely used for intelligent surveillance systems to detect moving targets by subtracting the static background components. Most roadside LiDAR object detection methods filter out foreground points by comparing new data points to pre-trained background references based on descriptive statistics over many frames (e.g., voxel density, number of neighbors, maximum distance). However, these solutions are inefficient under heavy traffic, and parameter values are hard to transfer from one scenario to another. In early studies, the probabilistic background modeling methods widely used for the video-based system were considered unsuitable for roadside LiDAR surveillance systems due to the sparse and unstructured point cloud data. In this paper, the raw LiDAR data were transformed into a structured representation based on the elevation and azimuth value of each LiDAR point. With this high-order tensor representation, we break the barrier to allow efficient high-dimensional multivariate analysis for roadside LiDAR background modeling. The Bayesian Nonparametric (BNP) approach integrates the intensity value and 3D measurements to exploit the measurement data using 3D and intensity info entirely. The proposed method was compared against two state-of-the-art roadside LiDAR background models, computer vision benchmark, and deep learning baselines, evaluated at point, object, and path levels under heavy traffic and challenging weather. This multimodal Weighted Bayesian Gaussian Mixture Model (GMM) can handle dynamic backgrounds with noisy measurements and substantially enhances the infrastructure-based LiDAR object detection, whereby various 3D modeling for smart city applications could be created.
This work aims to address the challenges in autonomous driving by focusing on the 3D perception of the environment using roadside LiDARs. We design a 3D object detection model that can detect traffic participants in roadside LiDARs in real-time. Our model uses an existing 3D detector as a baseline and improves its accuracy. To prove the effectiveness of our proposed modules, we train and evaluate the model on three different vehicle and infrastructure datasets. To show the domain adaptation ability of our detector, we train it on an infrastructure dataset from China and perform transfer learni…
University of California at Riverside（加州大学河滨分校）；Toyota North America R&D Labs（丰田北美研发实验室）
Object perception plays a fundamental role in Cooperative Driving Automation (CDA) which is regarded as a revolutionary promoter for the next-generation transportation systems. However, the vehicle-based perception may suffer from the limited sensing range and occlusion as well as low penetration rates in connectivity. In this paper, we propose Cyber Mobility Mirror (CMM), a next-generation real-time traffic surveillance system for 3D object perception and reconstruction, to explore the potential of roadside sensors for enabling CDA in the real world. The CMM system consists of six main compon…
University of California at Riverside（加州大学河滨分校）；Toyota North America R&D Labs（丰田北美研发实验室）
3D object detection plays a fundamental role in enabling autonomous driving, which is regarded as the significant key to unlocking the bottleneck of contemporary transportation systems from the perspectives of safety, mobility, and sustainability. Most of the state-of-the-art (SOTA) object detection methods from point clouds are developed based on a single onboard LiDAR, whose performance will be inevitably limited by the range and occlusion, especially in dense traffic scenarios. In this paper, we propose \textit{PillarGrid}, a novel cooperative perception method fusing information from multi…
Rutgers, The State University of New Jersey（罗格斯大学，新泽西州立大学）
In this paper, we developed the solution of roadside LiDAR object detection using a combination of two unsupervised learning algorithms. The 3D point clouds are firstly converted into spherical coordinates and filled into the elevation-azimuth matrix using a hash function. After that, the raw LiDAR data were rearranged into new data structures to store the information of range, azimuth, and intensity. Then, the Dynamic Mode Decomposition method is applied to decompose the LiDAR data into low-rank backgrounds and sparse foregrounds based on intensity channel pattern recognition. The Coarse Fine…
VanJee Technology（万集科技）；BeiJing Academy of Artificial Intelligence（北京人工智能研究院）
As the roadside perception plays an increasingly significant role in the Connected Automated Vehicle Highway(CAVH) technologies, there are immediate needs of challenging real-world roadside datasets for bench marking and training various computer vision tasks such as 2D/3D object detection and multi-sensor fusion. In this paper, we firstly introduce a challenging BAAI-VANJEE roadside dataset which consist of LiDAR data and RGB images collected by VANJEE smart base station placed on the roadside about 4.5m high. This dataset contains 2500 frames of LiDAR data, 5000 frames of RGB images, includi…
RS2AD-LiDAR: End-to-End Autonomous Driving LiDAR Data Generation from Roadside Sensor Observations
State Key Laboratory of Intelligent Green Vehicle and Mobility（智能绿色车辆与移动国家重点实验室）；School of Vehicle and Mobility（车辆与移动学院）；College of Artificial Intelligence（人工智能学院）；Logic & Silicon AI Studio；School of Instrumentation and Optoelectronic Engineering（仪器科学与光电工程学院）
A Fully Interpretable Statistical Approach for Roadside LiDAR Background Subtraction
Fundación Vicomtech, Basque Research and Technology Alliance (BRTA)（维克森科技基金会、巴斯克研究与技术联盟）；University of the Basque Country (UPV/EHU)（巴斯克国家大学）；CAFSignalling（CAF信号公司）；IKERBASQUE, Basque Foundation for Science（伊基尔巴斯克基金会）；Donostia International Physics Center (DIPC)（多诺西亚国际物理中心）；Biofisika Institute（生物物理研究所）
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（自动化与智能感知学院，上海交通大学）；Global College, Shanghai Jiao Tong University（全球学院，上海交通大学）
LiDAR-based roadside perception is a cornerstone of advanced Intelligent Transportation Systems (ITS). While considerable research has addressed optimal LiDAR placement for infrastructure, the profound impact of differing LiDAR scanning patterns on perceptual performance remains comparatively under-investigated. The inherent nature of various scanning modes - such as traditional repetitive (mechanical/solid-state) versus emerging non-repetitive (e.g. prism-based) systems - leads to distinct point cloud distributions at varying distances, critically dictating the efficacy of object detection an…
Beam-Wise Statistical Background Subtraction for Static Roadside LiDAR: A Cross-Sensor Benchmark Study
University of Applied Sciences Esslingen（埃斯林根应用科学大学）；Faculty of Computer Science and Engineering（计算机科学与工程学院）；Institute for Intelligent Systems（智能系统研究所）
RS2AD: End-to-End Autonomous Driving Data Generation from Roadside Sensor Observations
School of Vehicle and Mobility, Tsinghua University（清华大学车辆与移动系统学院）；State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University（清华大学智能绿色车辆与移动系统国家重点实验室）；School of Instrumentation and Optoelectronic Engineering, BeiHang University（北航仪器与光电工程学院）；Department of Automation, Tsinghua University（清华大学自动化系）
Enhancing Highway Safety: Accident Detection on the A9 Test Stretch Using Roadside Sensors
Technical University of Munich (TUM)（慕尼黑工业大学）；University of California Merced (UCM)（加利福尼亚大学默塞德分校）；Fraunhofer Institute for Transportation and Infrastructure Systems (IVI)（弗劳恩霍夫交通与基础设施系统研究所）；SETLabs Research GmbH（SETLabs 研究有限公司）
AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection
Fujian Key Laboratory of Urban Intelligent Sensing and Computing（福建城市智能感知与计算重点实验室）；School of Informatics, Xiamen University（厦门大学信息学院）；Zhongguancun Academy（中关村学院）
Carnegie Mellon University（卡内基梅隆大学）；Bosch Research and Technology Center（博世研究与技术中心）
mmWave radars offer excellent depth resolution even at very long ranges owing to their high bandwidth. But their angular resolution is at least an order-of-magnitude worse than camera and lidar systems. Hence, mmWave radar is not a capable 3-D imaging solution in isolation. We propose Metamoran, a system that combines the complimentary strengths of radar and camera to obtain accurate, high resolution depth images over long ranges even in high clutter environments, all from a single fixed vantage point. Metamoran enables rich long-range depth imaging with applications in security and surveillan…
SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection
AICV Lab, University of Arkansas, USA（AICV实验室，阿肯色大学，美国）；University of Utah, USA（犹他大学，美国）；University of Houston, USA（休斯顿大学，美国）；University of Liverpool, UK（利物浦大学，英国）
LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；National University of Singapore（新加坡国立大学）；Shanghai AI Laboratory（上海人工智能实验室）；Nanjing University of Posts and Telecommunications（南京邮电大学）；S-Lab, Nanyang Technological University（南洋理工大学S-Lab）
Visual data in autonomous driving perception, such as camera image and LiDAR point cloud, can be interpreted as a mixture of two aspects: semantic feature and geometric structure. Semantics come from the appearance and context of objects to the sensor, while geometric structure is the actual 3D shape of point clouds. Most detectors on LiDAR point clouds focus only on analyzing the geometric structure of objects in real 3D space. Unlike previous works, we propose to learn both semantic feature and geometric structure via a unified multi-view framework. Our method exploits the nature of LiDAR sc…
Centre for Advanced Robotics Technology Innovation (CARTIN)（高级机器人技术创新中心（CARTIN））；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）
Data-intensive machine learning based techniques increasingly play a prominent role in the development of future mobility solutions - from driver assistance and automation functions in vehicles, to real-time traffic management systems realized through dedicated infrastructure. The availability of high quality real-world data is often an important prerequisite for the development and reliable deployment of such systems in large scale. Towards this endeavour, we present the A9-Dataset based on roadside sensor infrastructure from the 3 km long Providentia++ test field near Munich in Germany. The …
This paper presents a robust probabilistic point registration method for estimating the rigid transformation (i.e. rotation matrix and translation vector) between two pointcloud dataset. The method improves the robustness of point registration and consequently the robot localization in the presence of outliers in the pointclouds which always occurs due to occlusion, dynamic objects, and sensor errors. The framework models the point registration task based on directional statistics on a unit sphere. In particular, a Kent distribution mixture model is adopted and the process of point registratio…
The ability of an autonomous vehicle to perform 3D tracking is essential for safe planing and navigation in cluttered environments. The main challenges for multi-object tracking (MOT) in autonomous driving applications reside in the inherent uncertainties regarding the number of objects, when and where the objects may appear and disappear, and uncertainties regarding objects' states. Random finite set (RFS) based approaches can naturally model these uncertainties accurately and elegantly, and they have been widely used in radar-based tracking applications. In this work, we developed an RFS-bas…
Point cloud registration sits at the core of many important and challenging 3D perception problems including autonomous navigation, SLAM, object/scene recognition, and augmented reality. In this paper, we present a new registration algorithm that is able to achieve state-of-the-art speed and accuracy through its use of a hierarchical Gaussian Mixture Model (GMM) representation. Our method constructs a top-down multi-scale representation of point cloud data by recursively running many small-scale data likelihood segmentations in parallel on a GPU. We leverage the resulting representation using …
U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequences
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；National University of Singapore（新加坡国立大学）；Fudan University（复旦大学）；S-Lab, Nanyang Technological University（南洋理工大学S实验室）；Nanjing University of Posts and Telecommunications（南京邮电大学）；SKL-TI
EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media
Center for Modeling, Simulation, & Imaging in Medicine（建模、仿真与成像医学中心）；Rensselaer Polytechnic Institute（伦斯勒理工学院）；University at Albany（阿尔巴尼大学）；IBM T.J. Watson Research Center（IBM 沃森研究中心）
Uncertainty Estimation and Out-of-Distribution Detection for LiDAR Scene Semantic Segmentation
Institute of Cartography and Geoinformatics, Leibniz University Hannover（汉诺威莱布尼茨大学制图与地理信息研究所）；Institute of Photogrammetry and GeoInformation, Leibniz University Hannover（汉诺威莱布尼茨大学摄影测量与地理信息研究所）
TrackAny3D: Transferring Pretrained 3D Models for Category-unified 3D Point Cloud Tracking
Zhejiang University of Technology（浙江工业大学）；RMIT University（皇家墨尔本理工大学）；Zhejiang Key Laboratory of Visual Information Intelligent Processing（浙江省视觉信息智能处理重点实验室）
