Monitoring the integrity of object detection for errors within the perception module of automated driving systems (ADS) is paramount for ensuring safety. Despite recent advancements in deep neural network (DNN)-based object detectors, their susceptibility to detection errors, particularly in the less-explored realm of 3D object detection, remains a significant concern. State-of-the-art integrity monitoring (also known as introspection) mechanisms in 2D object detection mainly utilise the activation patterns in the final layer of the DNN-based detector's backbone. However, that may not sufficiently address the complexities and sparsity of data in 3D object detection. To this end, we conduct, in this article, an extensive investigation into the effects of activation patterns extracted from various layers of the backbone network for introspecting the operation of 3D object detectors. Through a comparative analysis using Kitti and NuScenes datasets with PointPillars and CenterPoint detectors, we demonstrate that using earlier layers' activation patterns enhances the error detection performance of the integrity monitoring system, yet increases computational complexity. To address the real-time operation requirements in ADS, we also introduce a novel introspection method that combines activation patterns from multiple layers of the detector's backbone and report its performance.
Monitoring and Diagnosability of Perception Systems
Massachusetts Institute of Technology(麻省理工学院)
感知是机器人和自主系统(例如自动驾驶车辆)高完整性应用的关键组成部分。在这些应用中,感知系统的故障可能会危及人类生命,广泛采用这些技术需要开发方法来保证和监测安全操作。尽管感知系统至关重要,但目前还没有正式的系统级监控方法。在这项工作中,我们提出了一个数学模型的感知系统的运行时监测和故障检测和识别。为了实现这一目标,我们绘制连接与文献中的多处理器系统的可诊断性,并将其推广到考虑随着时间的推移与异构的输出交互的模块。由此产生的时间诊断图(i)提供了一个框架,原因的一致性的感知输出-跨模块和随着时间的推移-从而使故障检测,(ii)使我们能够建立正式的保证最大数量的故障,可以唯一地确定在一个给定的感知系统,和(iii)使设计有效的算法进行故障识别。我们使用LGSVL自动驾驶模拟器和Apollo Auto autonomy软件堆栈在现实模拟中演示了我们的监控系统PerSyS,并表明PerSyS能够在具有挑战性的场景中检测故障(包括近年来导致自动驾驶汽车事故的场景),并且能够正确识别故障,同时需要最小的计算开销(单核CPU上<5 ms)。
A Knowledge-Driven Diffusion Policy for End-to-End Autonomous Driving Based on Expert Routing
College of Transportation, Tongji University(同济大学交通运输学院);Department of Computer Science, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校计算机科学系)
Integrity Monitoring of 3D Object Detection in Automated Driving Systems using Raw Activation Patterns and Spatial Filtering
WMG, University of Warwick(华威大学WMG学院);School of Electronics, Electrical Engineering and Computer Science (EEECS), Queen’s University of Belfast(贝尔法斯特女王大学电子、电气工程与计算机科学学院)
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));The Hong Kong University of Science and Technology(香港科技大学);Tsinghua University(清华大学);Sichuan Agricultural University(四川农业大学)
AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Shenzhen Research Institute of Big Data(深圳大数据研究院);Sun Yat-sen University(中山大学);Baidu Inc.(百度公司);Guangdong Key Laboratory of Big Data Analysis and Processing(广东大数据分析与处理重点实验室)
Balancing Robustness and Efficiency in Embedded DNNs Through Activation Function Selection
Department of Electronics Technology, University of the Basque Country (UPV/EHU)(电子技术系,巴斯克国家大学(UPV/EHU));Department of Electricity and Electronics, University of the Basque Country (UPV/EHU)(电力与电子系,巴斯克国家大学(UPV/EHU))
AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception
Fudan University(复旦大学);Institute of Meta-Medical(元医学研究所);Engineering Research Center of AI and Robotics, Ministry of Education(教育部人工智能与机器人工程研究中心);AI and Unmanned Systems Engineering Research Center of Jilin Province(吉林省人工智能与无人系统工程研究中心);Boli Technology Co., Ltd.(博力科技有限公司)
DriftNet: Aggressive Driving Behavior Classification using 3D EfficientNet Architecture
Prince Sultan University(苏丹王子大学);INESC-TEC(波尔图理工学院高等工程研究所技术与科学创新研究所);Polytechnic Institute of Porto(波尔图理工学院)
侵略性驾驶(即,汽车漂移)是使人的安全和生命处于重大风险中的危险行为。这种行为被认为是关于公共交通道路中的常规交通的异常。深度学习的最新技术提出了在不同背景下进行异常检测的新方法,例如行人监控、街头斗殴和威胁检测。在本文中,我们提出了一个新的异常检测框架,适用于检测侵略性驾驶行为。我们的贡献在于开发一个3D神经网络架构,基于最先进的EfficientNet 2D图像分类器,用于视频中的侵略性驾驶检测。我们提出了一个有效的Net 3D CNN特征提取器的视频分析,我们比较它与现有的特征提取器。我们还创建了沙特阿拉伯背景下的汽车漂移数据集https://www.youtube.com/watch? v=vLzgye1-d1k。据我们所知,这是第一个使用深度学习解决攻击性驾驶行为问题的工作。