Just as humans can become disoriented in featureless deserts or thick fogs, not all environments are conducive to the Localization Accuracy and Stability (LAS) of autonomous robots. This paper introduces an efficient framework designed to enhance LiDAR-based LAS through strategic trajectory generation, known as Perception-aware Planning. Unlike vision-based frameworks, the LiDAR-based requires different considerations due to unique sensor attributes. Our approach focuses on two main aspects: firstly, assessing the impact of LiDAR observations on LAS. We introduce a perturbation-induced metric to provide a comprehensive and reliable evaluation of LiDAR observations. Secondly, we aim to improve motion planning efficiency. By creating a Static Observation Loss Map (SOLM) as an intermediary, we logically separate the time-intensive evaluation and motion planning phases, significantly boosting the planning process. In the experimental section, we demonstrate the effectiveness of the proposed metrics across various scenes and the feature of trajectories guided by different metrics. Ultimately, our framework is tested in a real-world scenario, enabling the robot to actively choose topologies and orientations preferable for localization. The source code is accessible at https://github.com/ZJU-FAST-Lab/LF-3PM.
University of Hong Kong（香港大学）；Harbin Institute of Technology（哈尔滨工业大学）
Recent advances in LiDAR technology have opened up new possibilities for robotic navigation. Given the widespread use of occupancy grid maps (OGMs) in robotic motion planning, this paper aims to address the challenges of integrating LiDAR with OGMs. To this end, we propose ROG-Map, a uniform grid-based OGM that maintains a local map moving along with the robot to enable efficient map operation and reduce memory costs for large-scene autonomous flight. Moreover, we present a novel incremental obstacle inflation method that significantly reduces the computational cost of inflation. The proposed …
DPNet: Doppler LiDAR Motion Planning for Highly-Dynamic Environments
The University of Hong Kong（香港大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究所）；University of Macau（澳门大学）；D-Robotics；Zhejiang University（浙江大学）
Socially Aware Motion Planning for Service Robots Using LiDAR and RGB-D Camera
Fulbright University Vietnam（富布赖特越南大学）；Vietnam National University Hanoi（越南国家大学河内分校）；University of Technology Sydney（悉尼科技大学）；Le Quy Don University（黎贵敦大学）
Oxford Robotics Institute, University of Oxford（牛津大学牛津机器人研究所）；Cluster of Excellence PhenoRob, Institute of Geodesy and Geoinformation, University of Bonn（波恩大学大地测量与地理信息研究所PhenoRob卓越集群）
Safe motion planning in robotics requires planning into space which has been verified to be free of obstacles. However, obtaining such environment representations using lidars is challenging by virtue of the sparsity of their depth measurements. We present a learning-aided 3D lidar reconstruction framework that upsamples sparse lidar depth measurements with the aid of overlapping camera images so as to generate denser reconstructions with more definitively free space than can be achieved with the raw lidar measurements alone. We use a neural network with an encoder-decoder structure to predict…
Omni-Perception: Omnidirectional Collision Avoidance for Legged Locomotion in Dynamic Environments
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Spatialtemporal AI（时空AI）；The Hong Kong University of Science and Technology（香港科技大学）；Department of Eletronic Engineering, Tsinghua University（清华大学电子工程系）；Beijing Innovation Center of Humanoid Robotics Co., Ltd.（北京人形机器人创新中心有限公司）
Robotics Research Center, IIIT Hyderabad（国际信息技术研究所海得拉巴分校机器人研究中心）；TCS Research（塔塔咨询服务公司研究部）
Table-top Rearrangement and Planning is a challenging problem that relies heavily on an excellent perception stack. The perception stack involves observing and registering the 3D scene on the table, detecting what objects are on the table, and how to manipulate them. Consequently, it greatly influences the system's task-planning and motion-planning stacks that follow. We present a comprehensive overview and discuss the different challenges associated with the perception module. This work is a result of our extensive involvement in the ICRA 2022 Open Cloud Robot Table Organization Challenge, in…
In this paper we describe a framework towards computing well-localized, robust motion plans through the perception-aware motion planning problem, whereby we seek a low-cost motion plan subject to a separate constraint on perception localization quality. To solve this problem we introduce the Multiobjective Perception-Aware Planning (MPAP) algorithm which explores the state space via a multiobjective search, considering both cost and a perception heuristic. This framework can accommodate a large range of heuristics, allowing those that capture the history dependence of localization drift and re…
This paper addresses a multi-robot planning problem in environments with partially unknown semantics. The environment is assumed to have known geometric structure (e.g., walls) and to be occupied by static labeled landmarks with uncertain positions and classes. This modeling approach gives rise to an uncertain semantic map generated by semantic SLAM algorithms. Our goal is to design control policies for robots equipped with noisy perception systems so that they can accomplish collaborative tasks captured by global temporal logic specifications. To specify missions that account for environmenta…
Most existing motion planning algorithms assume that a map (of some quality) is fully determined prior to generating a motion plan. In many emerging applications of robotics, e.g., fast-moving agile aerial robots with constrained embedded computational platforms and visual sensors, dense maps of the world are not immediately available, and they are computationally expensive to construct. We propose a new algorithm for generating plan graphs which couples the perception and motion planning processes for computational efficiency. In a nutshell, the proposed algorithm iteratively switches between…
AeroTraj: Trajectory Planning for Fast, and Accurate 3D Reconstruction Using a Drone-based LiDAR
Rochester Institute of Technology（罗切斯特理工学院）；University of Southern California（南加州大学）；Nokia Bell Labs（诺基亚贝尔实验室）；Georgia Institute of Technology（佐治亚理工学院）
The University of Texas at Austin（德克萨斯大学奥斯汀分校）；The University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）；Alliance Innovation Lab Silicon Valley（硅谷联盟创新实验室）
Robots deployed in the real world over extended periods of time need to reason about unexpected failures, learn to predict them, and to proactively take actions to avoid future failures. Existing approaches for competence-aware planning are either model-based, requiring explicit enumeration of known failure modes, or purely statistical, using state- and location-specific failure statistics to infer competence. We instead propose a structured model-free approach to competence-aware planning by reasoning about plan execution failures due to errors in perception, without requiring a priori enumer…
Istanbul Technical University（伊斯坦布尔理工大学）；Turkish-German University（土耳其-德国大学）
We investigate how to utilize predictive models for selecting appropriate motion planning strategies based on perception uncertainty estimation for agile unmanned aerial vehicle (UAV) navigation tasks. Although there are variety of motion planning and perception algorithms for such tasks, the impact of perception uncertainty is not explicitly handled in many of the current motion algorithms, which leads to performance loss in real-life scenarios where the measurement are often noisy due to external disturbances. We develop a novel framework for embedding perception uncertainty to high level mo…
Autonomous vehicles rely on their perception systems to acquire information about their immediate surroundings. It is necessary to detect the presence of other vehicles, pedestrians and other relevant entities. Safety concerns and the need for accurate estimations have led to the introduction of Light Detection and Ranging (LiDAR) systems in complement to the camera or radar-based perception systems. This article presents a review of state-of-the-art automotive LiDAR technologies and the perception algorithms used with those technologies. LiDAR systems are introduced first by analyzing the mai…
The Hong Kong University of Science and Technology（香港科技大学）；Chinese Academy of Sciences（中国科学院）
High precision 3D LiDARs are still expensive and hard to acquire. This paper presents the characteristics of RS-LiDAR, a model of low-cost LiDAR with sufficient supplies, in comparison with VLP-16. The paper also provides a set of evaluations to analyze the characterizations and performances of LiDARs sensors. This work analyzes multiple properties, such as drift effects, distance effects, color effects and sensor orientation effects, in the context of 3D perception. By comparing with Velodyne LiDAR, we found RS-LiDAR as a cheaper and acquirable substitute of VLP-16 with similar efficiency.
ROS 2-Based LiDAR Perception Framework for Mobile Robots in Dynamic Production Environments, Utilizing Synthetic Data Generation, Transformation-Equivariant 3D Detection and Multi-Object Tracking
Chair of Intelligence in Quality Sensing (IQS), Laboratory for Machine Tools and Production Engineering (WZL), RWTH Aachen University（亚琛工业大学机床与生产工程实验室质量感知智能教席）；The Sirindhorn International Thai-German Graduate School of Engineering (TGGS), King Mongkut’s University of Technology North Bangkok (KMUTNB)（北曼谷先皇技术大学泰德国际工程研究生院）
LiDAR sensors are a powerful tool for robot simultaneous localization and mapping (SLAM) in unknown environments, but the raw point clouds they produce are dense, computationally expensive to store, and unsuited for direct use by downstream autonomy tasks, such as motion planning. For integration with motion planning, it is desirable for SLAM pipelines to generate lightweight geometric map representations. Such representations are also particularly well-suited for man-made environments, which can often be viewed as a so-called "Manhattan world" built on a Cartesian grid. In this work we presen…
PERAL: Perception-Aware Motion Control for Passive LiDAR Excitation in Spherical Robots
Centre for Advanced Robotics Technology Innovation (CARTIN), School of Electrical and Electronic Engineering, Nanyang Technological University（先进机器人技术创新中心（CARTIN）、电气与电子工程学院、南洋理工大学）
Applications involving autonomous navigation and planning of mobile agents can benefit greatly by employing online Simultaneous Localization and Mapping (SLAM) techniques, however, their proper implementation still warrants an efficient amalgamation with any offline path planning method that may be used for the particular application. In this paper, such a case of amalgamation is considered for a LiDAR-based indoor mapping system which presents itself as a 2D coverage path planning problem implemented along with online SLAM. This paper shows how classic offline Coverage Path Planning (CPP) can…
We present a LiDAR-based and real-time capable 3D perception system for automated driving in urban domains. The hierarchical system design is able to model stationary and movable parts of the environment simultaneously and under real-time conditions. Our approach extends the state of the art by innovative in-detail enhancements for perceiving road users and drivable corridors even in case of non-flat ground surfaces and overhanging or protruding elements. We describe a runtime-efficient pointcloud processing pipeline, consisting of adaptive ground surface estimation, 3D clustering and motion c…
ParisLuco3D: A high-quality target dataset for domain generalization of LiDAR perception
Centre for Robotics, Mines Paris - PSL, PSL University（巴黎文理研究大学、巴黎矿业学院机器人中心）；U2IS, ENSTA Paris, Institut Polytechnique de Paris（巴黎综合理工学院、巴黎国立高等先进技术学院U2IS实验室）
HumanoidPano: Hybrid Spherical Panoramic-LiDAR Cross-Modal Perception for Humanoid Robots
Beijing Innovation Center of Humanoid Robotics（北京人形机器人创新中心）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Hong Kong University of Science and Technology（香港科技大学）；Beijing Institute of Technology（北京理工大学）；ETH Zurich（苏黎世联邦理工学院）；Nanyang Technological University（南洋理工大学）；X-Humanoid（人形智能科技公司）
State Key Laboratory of Internet of Things for Smart City (SKL-IOTSC)（物联网智能城市国家重点实验室）；Faculty of Science and Technology（科技学院）；University of Macau（澳门大学）；School of Computing and Information Systems（计算与信息学院）；Singapore Management University（新加坡管理大学）；Department of Naval Architecture and Marine Engineering and Department of Robotics（船舶工程与海洋工程系和机器人系）；University of Michigan（密歇根大学）；Department of Robotics（机器人系）；Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)（穆罕默德·本·拉希德智能人工智能大学）
Robust perception is an essential component to enable long-term operation of mobile robots. It depends on failure resilience through reliable sensor data and preprocessing, as well as failure awareness through introspection, for example the ability to self-assess localization performance. This paper presents CorAl: a principled, intuitive, and generalizable method to measure the quality of alignment between pairs of point clouds, which learns to detect alignment errors in a self-supervised manner. CorAl compares the differential entropy in the point clouds separately with the entropy in their …
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University（自动化与智能感知学院，上海交通大学）；Global College, Shanghai Jiao Tong University（全球学院，上海交通大学）
LiDAR-based roadside perception is a cornerstone of advanced Intelligent Transportation Systems (ITS). While considerable research has addressed optimal LiDAR placement for infrastructure, the profound impact of differing LiDAR scanning patterns on perceptual performance remains comparatively under-investigated. The inherent nature of various scanning modes - such as traditional repetitive (mechanical/solid-state) versus emerging non-repetitive (e.g. prism-based) systems - leads to distinct point cloud distributions at varying distances, critically dictating the efficacy of object detection an…
Modern autonomous vehicles rely heavily on mechanical LiDARs for perception. Current perception methods generally require 360° point clouds, collected sequentially as the LiDAR scans the azimuth and acquires consecutive wedge-shaped slices. The acquisition latency of a full scan (~ 100ms) may lead to outdated perception which is detrimental to safe operation. Recent streaming perception works proposed directly processing LiDAR slices and compensating for the narrow field of view (FOV) of a slice by reusing features from preceding slices. These works, however, are all based on a single modality…
Generating Roadside LiDAR Datasets from Vehicle-Side Datasets via Novel View Synthesis
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China（自动化与智能感知学院，上海交通大学，上海200240，中国）；Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China（系统控制与信息处理重点实验室，中华人民共和国教育部，上海200240，中国）；Global College, Shanghai Jiao Tong University, Shanghai 200240, China（全球学院，上海交通大学，上海200240，中国）
University of Waterloo（滑铁卢大学）；University of Toronto（多伦多大学）
We introduce the Precise Synthetic Image and LiDAR (PreSIL) dataset for autonomous vehicle perception. Grand Theft Auto V (GTA V), a commercial video game, has a large detailed world with realistic graphics, which provides a diverse data collection environment. Existing works creating synthetic LiDAR data for autonomous driving with GTA V have not released their datasets, rely on an in-game raycasting function which represents people as cylinders, and can fail to capture vehicles past 30 metres. Our work creates a precise LiDAR simulator within GTA V which collides with detailed models for all…
Analyzing Infrastructure LiDAR Placement with Realistic LiDAR Simulation Library
Shanghai AI Laboratory（上海人工智能实验室）；Institute of Artificial Intelligence, Beihang University（北京航空航天大学人工智能研究院）；University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Lehigh University（利哈伊大学）；Eindhoven University of Technology（埃因霍温理工大学）
This work presents a step towards utilizing incrementally-improving symbolic perception knowledge of the robot's surroundings for provably correct reactive control synthesis applied to an autonomous driving problem. Combining abstract models of motion control and information gathering, we show that assume-guarantee specifications (a subclass of Linear Temporal Logic) can be used to define and resolve traffic rules for cautious planning. We propose a novel representation called symbolic refinement tree for perception that captures the incremental knowledge about the environment and embodies the…
In robotic applications, the control, and actuation deal with a continuous description of the system and environment, while high-level planning usually works with a discrete description. This paper considers the problem of bridging the low-level control and high-level planning for robotic systems via sensor data. In particular, we propose a discretization algorithm that identifies free polytopes via lidar point cloud data. A transition graph is then constructed where each node corresponds to a free polytope and two nodes are connected with an edge if the two corresponding free polytopes inters…
Virginia Tech（弗吉尼亚理工大学）；University of Maryland（马里兰大学）
We propose a risk-aware framework for multi-robot, multi-demand assignment and planning in unknown environments. Our motivation is disaster response and search-and-rescue scenarios where ground vehicles must reach demand locations as soon as possible. We consider a setting where the terrain information is available only in the form of an aerial, georeferenced image. Deep learning techniques can be used for semantic segmentation of the aerial image to create a cost map for safe ground robot navigation. Such segmentation may still be noisy. Hence, we present a joint planning and perception frame…
