In this paper, we present an Efficient Planning System for automated vehicles In highLy interactive envirONments (EPSILON). EPSILON is an efficient interaction-aware planning system for automated driving, and is extensively validated in both simulation and real-world dense city traffic. It follows a hierarchical structure with an interactive behavior planning layer and an optimization-based motion planning layer. The behavior planning is formulated from a partially observable Markov decision process (POMDP), but is much more efficient than naively applying a POMDP to the decision-making problem. The key to efficiency is guided branching in both the action space and observation space, which decomposes the original problem into a limited number of closed-loop policy evaluations. Moreover, we introduce a new driver model with a safety mechanism to overcome the risk induced by the potential imperfectness of prior knowledge. For motion planning, we employ a spatio-temporal semantic corridor (SSC) to model the constraints posed by complex driving environments in a unified way. Based on the SSC, a safe and smooth trajectory is optimized, complying with the decision provided by the behavior planner. We validate our planning system in both simulations and real-world dense traffic, and the experimental results show that our EPSILON achieves human-like driving behaviors in highly interactive traffic flow smoothly and safely without being over-conservative compared to the existing planning methods.
A Terrain-Adaptive epsilon-Constraint MPC for Uneven Terrain Kinodynamic Planning
Laboratório de Engenharia de Sistemas e Robótica, Universidade Federal da Paraíba（系统与机器人工程实验室，帕拉伊巴联邦大学）；School of Computer Science, University of Lincoln（计算机科学学院，林肯大学）
University of Utah（犹他大学）；University of Tennessee（田纳西大学）
Tendon-driven robots, where one or more tendons under tension bend and manipulate a flexible backbone, can improve minimally invasive surgeries involving difficult-to-reach regions in the human body. Planning motions safely within constrained anatomical environments requires accuracy and efficiency in shape estimation and collision checking. Tendon robots that employ arbitrarily-routed tendons can achieve complex and interesting shapes, enabling them to travel to difficult-to-reach anatomical regions. Arbitrarily-routed tendon-driven robots have unintuitive nonlinear kinematics. Therefore, we …
Mean Field Game-Based Interactive Trajectory Planning Using Physics-Inspired Unified Potential Fields
School of Computing Science, University of Glasgow（格拉斯哥大学计算机科学学院）；College of Mechanical Engineering, Chongqing University of Technology（重庆理工大学机械工程学院）；Laboratory of Intelligent Home Appliances, College of Science and Technology, Ningbo University（宁波大学智能家电实验室）
Collision-free mobile robot navigation is an important problem for many robotics applications, especially in cluttered environments. In such environments, obstacles can be static or dynamic. Dynamic obstacles can additionally be interactive, i.e. changing their behavior according to the behavior of other entities. The perception and prediction modules of robotic systems create probabilistic representations and predictions of such environments. In this paper, we propose a novel prediction representation for interactive behaviors of dynamic obstacles. Then, we propose a real-time trajectory plan…
University of Lincoln（林肯大学）；Centre for Autonomous Systems (L-CAS)（自主系统中心（L-CAS））
Robotic technology is increasingly considered the major mean for fruit picking. However, picking fruits in a dense cluster imposes a challenging research question in terms of motion/path planning as conventional planning approaches may not find collision-free movements for the robot to reach-and-pick a ripe fruit within a dense cluster. In such cases, the robot needs to safely push unripe fruits to reach a ripe one. Nonetheless, existing approaches to planning pushing movements in cluttered environments either are computationally expensive or only deal with 2-D cases and are not suitable for f…
Many robotic applications involve interactions between multiple agents where an agent's decisions affect the behavior of other agents. Such behaviors can be captured by the equilibria of differential games which provide an expressive framework for modeling the agents' mutual influence. However, finding the equilibria of differential games is in general challenging as it involves solving a set of coupled optimal control problems. In this work, we propose to leverage the special structure of multi-agent interactions to generate interactive trajectories by simply solving a single optimal control …
Motion planning for autonomous robots and vehicles in presence of uncontrolled agents remains a challenging problem as the reactive behaviors of the uncontrolled agents must be considered. Since the uncontrolled agents usually demonstrate multimodal reactive behavior, the motion planner needs to solve a continuous motion planning problem under these behaviors, which contains a discrete element. We propose a branch Model Predictive Control (MPC) framework that plans over feedback policies to leverage the reactive behavior of the uncontrolled agent. In particular, a scenario tree is constructed …
Quadruped platforms have become an active topic of research due to their high mobility and traversability in rough terrain. However, it is highly challenging to determine whether the clattered environment could be passed by the robot and how exactly its path should be calculated. Moreover, the calculated path may pass through areas with dynamic objects or environments that are dangerous for the robot or people around. Therefore, we propose a novel conceptual approach of teaching quadruped robots navigation through user-guided path planning in virtual reality (VR). Our system contains both glob…
Fast and Scalable Game-Theoretic Trajectory Planning with Intentional Uncertainties
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；The Hong Kong University of Science and Technology（香港科学与技术大学）；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科学与技术大学深圳-香港协同创新研究院）
Autonomous navigation in dense traffic scenarios remains challenging for autonomous vehicles (AVs) because the intentions of other drivers are not directly observable and AVs have to deal with a wide range of driving behaviors. To maneuver through dense traffic, AVs must be able to reason how their actions affect others (interaction model) and exploit this reasoning to navigate through dense traffic safely. This paper presents a novel framework for interaction-aware motion planning in dense traffic scenarios. We explore the connection between human driving behavior and their velocity changes w…
AINav: Large Language Model-Based Adaptive Interactive Navigation
School of Advanced Manufacturing and Robotics, Peking University（北京大学先进制造与机器人学院）；AI Institute, School of Computer Science, Shanghai Jiao Tong University（上海交通大学计算机学院人工智能研究所）
Humans exhibit incredibly high levels of multi-modal understanding - combining visual cues with read, or heard knowledge comes easy to us and allows for very accurate interaction with the surrounding environment. Various simulation environments focus on providing data for tasks related to scene understanding, question answering, space exploration, visual navigation. In this work, we are providing a solution to encompass both, visual and behavioural aspects of simulation in a new environment for learning interactive reasoning in manipulation setup. SAMPLE-HD environment allows to generate vario…
Point cloud has been widely used in the field of autonomous driving since it can provide a more comprehensive three-dimensional representation of the environment than 2D images. Point-wise prediction based on point cloud sequence (PCS) is an essential part of environment understanding, which can assist in the decision-making and motion-planning of autonomous vehicles. However, PCS prediction has not been deeply researched in the literature. This paper proposes a brief review of the sequential point cloud prediction methods, focusing on interactive scenarios. Firstly, we define the PCS predicti…
fortiss GmbH（福特斯有限公司）；Technische Universität München（慕尼黑工业大学）
Autonomous vehicles need to abide by the same rules that humans follow. Some of these traffic rules may depend on multiple agents or time. Especially in situations with traffic participants that interact densely, the interactions with other agents need to be accounted for during planning. To study how multi-agent and time-dependent traffic rules shall be modeled, a framework is needed that restricts the behavior to rule-conformant actions during planning, and that can eventually evaluate the satisfaction of these rules. This work presents a method to model the conformance to traffic rules for …
From Dialogue to Execution: Mixture-of-Agents Assisted Interactive Planning for Behavior Tree-Based Long-Horizon Robot Execution
Waseda University（早稻田大学）；National Institute of Informatics（国家信息研究所）；NII Research and Development Center for Large Language Models（国家信息研究所大型语言模型研究开发中心）；The Graduate University for Advanced Studies (SOKENDAI)（高级研究大学研究生院（SOKENDAI））
The Hong Kong University of Science and Technology（香港科技大学）；University of California San Diego（加利福尼亚大学圣迭戈分校）；The Hong Kong Polytechnic University（香港理工大学）
With the recent advancement of deep learning technology, data-driven approaches for autonomous car prediction and planning have achieved extraordinary performance. Nevertheless, most of these approaches follow a non-interactive prediction and planning paradigm, hypothesizing that a vehicle's behaviors do not affect others. The approaches based on such a non-interactive philosophy typically perform acceptably in sparse traffic scenarios but can easily fail in dense traffic scenarios. Therefore, we propose an end-to-end interactive neural motion planner (INMP) for autonomous driving in this pape…
Interactive Motion Planning for Human-Robot Collaboration Based on Human-Centric Configuration Space Ergonomic Field
Department of Mechanical and Automation Engineering, T-Stone Robotics Institute, The Chinese University of Hong Kong（机械与自动化工程系，T-Stone机器人研究所，香港中文大学）；Idiap Research Institute（Idiap研究所）；Department of Advanced Robotics, Istituto Italiano di Tecnologia（先进机器人系，意大利Istituto Italiano di Tecnologia）
Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling
Institute for AI Industry Research (AIR), Tsinghua University（人工智能产业研究所（AIR），清华大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；The Chinese University of Hong Kong（香港中文大学）
Neural network-based driving planners have shown great promises in improving task performance of autonomous driving. However, it is critical and yet very challenging to ensure the safety of systems with neural network based components, especially in dense and highly interactive traffic environments. In this work, we propose a safety-driven interactive planning framework for neural network-based lane changing. To prevent over conservative planning, we identify the driving behavior of surrounding vehicles and assess their aggressiveness, and then adapt the planned trajectory for the ego vehicle …
In most classical Autonomous Vehicle (AV) stacks, the prediction and planning layers are separated, limiting the planner to react to predictions that are not informed by the planned trajectory of the AV. This work presents a module that tightly couples these layers via a game-theoretic Model Predictive Controller (MPC) that uses a novel interactive multi-agent neural network policy as part of its predictive model. In our setting, the MPC planner considers all the surrounding agents by informing the multi-agent policy with the planned state sequence. Fundamental to the success of our method is …
Affordance-Aware Interactive Decision-Making and Execution for Ambiguous Instructions
Department of Automation, Tsinghua University（清华大学自动化系）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息学院）；Computer Science and Engineering Department, Chinese University of Hong Kong（香港中文大学计算机科学与工程系）
Tsinghua University（清华大学）；Shanghai Qi Zhi Institute（上海期智研究院）；Massachusetts Institute of Technology（麻省理工学院）
Motion prediction is crucial in enabling safe motion planning for autonomous vehicles in interactive scenarios. It allows the planner to identify potential conflicts with other traffic agents and generate safe plans. Existing motion predictors often focus on reducing prediction errors, yet it remains an open question on how well they help identify the conflicts for the planner. In this paper, we evaluate state-of-the-art predictors through novel conflict-related metrics, such as the success rate of identifying conflicts. Surprisingly, the predictors suffer from a low success rate and thus lead…
IR-STP: Enhancing Autonomous Driving with Interaction Reasoning in Spatio-Temporal Planning
The Hong Kong University of Science and Technology（香港科技大学）；HKUST (GZ)（香港科技大学（广州））；HKUST Shenzhen-Hong Kong Collaborative Innovation Research Institute（香港科技大学深港协同创新研究院）；Unity Drive Innovation Technology Co. Ltd.（元戎启行创新科技有限公司）
