感知是机器人和自主系统(例如自动驾驶车辆)高完整性应用的关键组成部分。在这些应用中,感知系统的故障可能会危及人类生命,广泛采用这些技术需要开发方法来保证和监测安全操作。尽管感知系统至关重要,但目前还没有正式的系统级监控方法。在这项工作中,我们提出了一个数学模型的感知系统的运行时监测和故障检测和识别。为了实现这一目标,我们绘制连接与文献中的多处理器系统的可诊断性,并将其推广到考虑随着时间的推移与异构的输出交互的模块。由此产生的时间诊断图(i)提供了一个框架,原因的一致性的感知输出-跨模块和随着时间的推移-从而使故障检测,(ii)使我们能够建立正式的保证最大数量的故障,可以唯一地确定在一个给定的感知系统,和(iii)使设计有效的算法进行故障识别。我们使用LGSVL自动驾驶模拟器和Apollo Auto autonomy软件堆栈在现实模拟中演示了我们的监控系统PerSyS,并表明PerSyS能够在具有挑战性的场景中检测故障(包括近年来导致自动驾驶汽车事故的场景),并且能够正确识别故障,同时需要最小的计算开销(单核CPU上<5 ms)。
英文摘要:
Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving vehicles. In these applications, failure of perception systems may put human life at risk, and a broad adoption of these technologies requires the development of methodologies to guarantee and monitor safe operation. Despite the paramount importance of perception systems, currently there is no formal approach for system-level monitoring. In this work, we propose a mathematical model for runtime monitoring and fault detection and identification in perception systems. Towards this goal, we draw connections with the literature on diagnosability in multiprocessor systems, and generalize it to account for modules with heterogeneous outputs that interact over time. The resulting temporal diagnostic graphs (i) provide a framework to reason over the consistency of perception outputs -- across modules and over time -- thus enabling fault detection, (ii) allow us to establish formal guarantees on the maximum number of faults that can be uniquely identified in a given perception system, and (iii) enable the design of efficient algorithms for fault identification. We demonstrate our monitoring system, dubbed PerSyS, in realistic simulations using the LGSVL self-driving simulator and the Apollo Auto autonomy software stack, and show that PerSyS is able to detect failures in challenging scenarios (including scenarios that have caused self-driving car accidents in recent years), and is able to correctly identify faults while entailing a minimal computation overhead (< 5 ms on a single-core CPU).
A holistic perception system of internal and external monitoring for ground autonomous vehicles: AutoTRUST paradigm
Industrial Systems Institute, ATHENA Research Center, Patras Science Park, Greece(工业系统研究所,ATHENA研究中心,帕特拉斯科学公园,希腊);AviSense.AI, Patras Science Park, Greece(AviSense.AI,帕特拉斯科学公园,希腊);Information Technologies Institute, CERTH, Greece(信息科技研究所,CERTH,希腊);KIOS Research and Innovation Center of Excellence, University of Cyprus, Cyprus(KIOS研究与创新中心,塞浦路斯大学,塞浦路斯)
AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception
Fudan University(复旦大学);Institute of Meta-Medical(元医学研究所);Engineering Research Center of AI and Robotics, Ministry of Education(教育部人工智能与机器人工程研究中心);AI and Unmanned Systems Engineering Research Center of Jilin Province(吉林省人工智能与无人系统工程研究中心);Boli Technology Co., Ltd.(博力科技有限公司)
School of Computing and Augmented Intelligence, Arizona State University(亚利桑那州立大学计算与增强智能学院);Department of Computer Science and Physics, Rider University(莱德大学计算机科学与物理系)
MTD: Multi-Timestep Detector for Delayed Streaming Perception
School of Computer and Electronic Information, Guangxi University(广西大学计算机与电子信息学院);Guangxi Intelligent Digital Services Research Center of Engineering Technology(广西智能数字服务工程技术研究中心)
Optimizing Indoor Farm Monitoring Efficiency Using UAV: Yield Estimation in a GNSS-Denied Cherry Tomato Greenhouse
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST)(机械工程系,乌山国立科学技术研究院);Agricultural Robotics and Artificial Intelligence, Metafarmers(农业机器人与人工智能,Metafarmers)
Sight Over Site: Perception-Aware Reinforcement Learning for Efficient Robotic Inspection
Robotics Systems Lab, ETH Zurich(苏黎世联邦理工学院机器人系统实验室);Computer Vision and Geometry Group, ETH Zurich(苏黎世联邦理工学院计算机视觉与几何组);Robotics and Perception Group, University of Zurich(苏黎世大学机器人与感知组);Microsoft Mixed Reality and AI Lab(微软混合现实与人工智能实验室)
Smart Roads: Roadside Perception, Vehicle-Road Cooperation and Business Model
Xidian University(西安电子科技大学);Guangzhou Institute of Technology, Xidian University(西安电子科技大学广州研究院);Eindhoven University of Technology(埃因霍温理工大学);Nanyang Technological University(南洋理工大学);University of Oslo(奥斯陆大学);Simula Metropolitan Center for Digital Engineering(西穆拉大都会数字工程中心)
GraphiContact: Pose-aware Human-Scene Robust Contact Perception for Interactive Systems
Tsinghua University, China(清华大学, 中国);XR System Application Research Center, Nanchang Research Institute, Zhejiang University, China(浙江大学南昌研究院XR系统应用研究中心, 中国);Beijing University of Posts and Telecommunications, China(北京邮电大学, 中国);University of Chinese Academy of Sciences, China(中国科学院大学, 中国);National University of Singapore, Singapore(新加坡国立大学, 新加坡)
Intermittent Deployment for Large-Scale Multi-Robot Forage Perception: Data Synthesis, Prediction, and Planning
Virginia Tech(弗吉尼亚理工大学);University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Oregon State University(俄勒冈州立大学);University of Maryland, College Park(马里兰大学帕克分校)
Scalable Radar-based Roadside Perception: Self-localization and Occupancy Heat Map for Traffic Analysis
Fraunhofer IVI(弗劳恩霍夫应用信息学研究所(IVI));Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院);Technische Hochschule Ingolstadt(英戈尔施塔特应用技术大学);Technical University of Munich(慕尼黑工业大学);Fraunhofer IOSB(弗劳恩霍夫光学、电子和图像利用研究所(IOSB))