Realistic pedestrian behaviour in the CARLA simulator using VR and mocap
University of Alcalá（阿尔卡拉大学）；European Commission, Joint Research Centre（欧盟委员会联合研究中心）
Sergio Martín Serrano, David Fernández Llorca, Iván García Daza, Miguel Ángel Sotelo Vázquez
备注：
This is a pre-print of the following work: Communications in Computer and Information Science (CCIS, volume 1882), 2023, Computer-Human Interaction Research and Applications reproduced with permission of Springer Nature. The final authenticated version is available online at: https://link.springer.com/chapter/10.1007/978-3-031-41962-1_5. arXiv admin note: substantial text overlap with arXiv:2206.00337
Simulations are gaining increasingly significance in the field of autonomous driving due to the demand for rapid prototyping and extensive testing. Employing physics-based simulation brings several benefits at an affordable cost, while mitigating potential risks to prototypes, drivers, and vulnerable road users. However, there exit two primary limitations. Firstly, the reality gap which refers to the disparity between reality and simulation and prevents the simulated autonomous driving systems from having the same performance in the real world. Secondly, the lack of empirical understanding regarding the behavior of real agents, such as backup drivers or passengers, as well as other road users such as vehicles, pedestrians, or cyclists. Agent simulation is commonly implemented through deterministic or randomized probabilistic pre-programmed models, or generated from real-world data; but it fails to accurately represent the behaviors adopted by real agents while interacting within a specific simulated scenario. This paper extends the description of our proposed framework to enable real-time interaction between real agents and simulated environments, by means immersive virtual reality and human motion capture systems within the CARLA simulator for autonomous driving. We have designed a set of usability examples that allow the analysis of the interactions between real pedestrians and simulated autonomous vehicles and we provide a first measure of the user's sensation of presence in the virtual environment.
ARCANE-PedSynth: Synthetic Multi-Pedestrian Datasets with Behavioural Crossing Annotations
Computer Vision Center (CVC), Universitat Aut\` o noma de Barcelona (UAB), Bellaterra, Barcelona, Spain；Institute of Electronics, Faculty of Computer Science, Electronics；Telecommunications, AGH University of Krakow, Krak\' o w, Poland
University of Alcalá（阿尔卡拉大学）；European Commission, Joint Research Centre (JRC)（欧盟委员会联合研究中心）
The role of simulation in autonomous driving is becoming increasingly important due to the need for rapid prototyping and extensive testing. The use of physics-based simulation involves multiple benefits and advantages at a reasonable cost while eliminating risks to prototypes, drivers and vulnerable road users. However, there are two main limitations. First, the well-known reality gap which refers to the discrepancy between reality and simulation that prevents simulated autonomous driving experience from enabling effective real-world performance. Second, the lack of empirical knowledge about …
Generating Realistic Safety-Critical Scenarios for Vehicle-Pedestrian Interactions
Transportation Informatics Lab, Department of Civil and Environmental Engineering, Old Dominion University（交通信息实验室，土木与环境工程系，旧 Dominion 大学）；Inner Mongolia Center for Transportation Research, Inner Mongolia University（内蒙古交通研究所，内蒙古大学）；School of Transportation and Logistics, National Engineering Laboratory of Integrated Transportation Big Data Application Technology, National and Local Joint Engineering Research Center of Integrated Transportation Intelligence, Southwest Jiaotong University（交通运输学院，国家集成交通大数据应用技术工程实验室，国家与地方联合集成交通智能工程研究中心，西南交通大学）
We propose a safe DRL approach for autonomous vehicle (AV) navigation through crowds of pedestrians while making a left turn at an unsignalized intersection. Our method uses two long-short term memory (LSTM) models that are trained to generate the perceived state of the environment and the future trajectories of pedestrians given noisy observations of their movement. A future collision prediction algorithm based on the future trajectories of the ego vehicle and pedestrians is used to mask unsafe actions if the system predicts a collision. The performance of our approach is evaluated in two exp…
Stellantis Group（斯泰兰蒂斯集团）；Université de Lorraine（洛林大学）；CNRS（法国国家科学研究中心）；Inria（法国国家信息与自动化研究所）
The human driver is no longer the only one concerned with the complexity of the driving scenarios. Autonomous vehicles (AV) are similarly becoming involved in the process. Nowadays, the development of AVs in urban places raises essential safety concerns for vulnerable road users (VRUs) such as pedestrians. Therefore, to make the roads safer, it is critical to classify and predict the pedestrians' future behavior. In this paper, we present a framework based on multiple variations of the Transformer models able to infer predict the pedestrian street-crossing decision-making based on the dynamics…
Saarland University（萨尔大学）；German Research Center for Artificial Intelligence（德国人工智能研究中心）
We present a novel hybrid learning method, HyLEAR, for solving the collision-free navigation problem for self-driving cars in POMDPs. HyLEAR leverages interposed learning to embed knowledge of a hybrid planner into a deep reinforcement learner to faster determine safe and comfortable driving policies. In particular, the hybrid planner combines pedestrian path prediction and risk-aware path planning with driving-behavior rule-based reasoning such that the driving policies also take into account, whenever possible, the ride comfort and a given set of driving-behavior rules. Our experimental perf…
HyPlan: Hybrid Learning-Assisted Planning Under Uncertainty for Safe Autonomous Driving
Saarland University, Computer Science Dept.（萨尔兰大学计算机科学系）；German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；French National Centre for Scientific Research (CNRS)（法国国家科学研究中心）
Urban autonomous driving in the presence of pedestrians as vulnerable road users is still a challenging and less examined research problem. This work formulates navigation in urban environments as a multi objective reinforcement learning problem. A deep learning variant of thresholded lexicographic Q-learning is presented for autonomous navigation amongst pedestrians. The multi objective DQN agent is trained on a custom urban environment developed in CARLA simulator. The proposed method is evaluated by comparing it with a single objective DQN variant on known and unknown environments. Evaluati…
LLM-based Realistic Safety-Critical Driving Video Generation
Department of Civil Engineering and Engineering Mechanics, Columbia University（哥伦比亚大学土木工程与工程力学系）；Department of Computer Science, Columbia University（哥伦比亚大学计算机科学系）
There are many artificial intelligence algorithms for autonomous driving, but directly installing these algorithms on vehicles is unrealistic and expensive. At the same time, many of these algorithms need an environment to train and optimize. Simulation is a valuable and meaningful solution with training and testing functions, and it can say that simulation is a critical link in the autonomous driving world. There are also many different applications or systems of simulation from companies or academies such as SVL and Carla. These simulators flaunt that they have the closest real-world simulat…
