Perception is a critical component of high-integrity applications of robotics and autonomous systems, such as self-driving cars. In these applications, failure of perception systems may put human life at risk, and a broad adoption of these technologies relies on the development of methodologies to guarantee and monitor safe operation as well as detect and mitigate failures. Despite the paramount importance of perception systems, currently there is no formal approach for system-level monitoring. In this work, we propose a mathematical model for runtime monitoring and fault detection of perception systems. Towards this goal, we draw connections with the literature on self-diagnosability for multiprocessor systems, and generalize it to (i) account for modules with heterogeneous outputs, and (ii) add a temporal dimension to the problem, which is crucial to model realistic perception systems where modules interact over time. This contribution results in a graph-theoretic approach that, given a perception system, is able to detect faults at runtime and allows computing an upper-bound on the number of faulty modules that can be detected. Our second contribution is to show that the proposed monitoring approach can be elegantly described with the language of topos theory, which allows formulating diagnosability over arbitrary time intervals.
Monitoring and Diagnosability of Perception Systems
Massachusetts Institute of Technology(麻省理工学院)
感知是机器人和自主系统(例如自动驾驶车辆)高完整性应用的关键组成部分。在这些应用中,感知系统的故障可能会危及人类生命,广泛采用这些技术需要开发方法来保证和监测安全操作。尽管感知系统至关重要,但目前还没有正式的系统级监控方法。在这项工作中,我们提出了一个数学模型的感知系统的运行时监测和故障检测和识别。为了实现这一目标,我们绘制连接与文献中的多处理器系统的可诊断性,并将其推广到考虑随着时间的推移与异构的输出交互的模块。由此产生的时间诊断图(i)提供了一个框架,原因的一致性的感知输出-跨模块和随着时间的推移-从而使故障检测,(ii)使我们能够建立正式的保证最大数量的故障,可以唯一地确定在一个给定的感知系统,和(iii)使设计有效的算法进行故障识别。我们使用LGSVL自动驾驶模拟器和Apollo Auto autonomy软件堆栈在现实模拟中演示了我们的监控系统PerSyS,并表明PerSyS能够在具有挑战性的场景中检测故障(包括近年来导致自动驾驶汽车事故的场景),并且能够正确识别故障,同时需要最小的计算开销(单核CPU上<5 ms)。
A holistic perception system of internal and external monitoring for ground autonomous vehicles: AutoTRUST paradigm
Industrial Systems Institute, ATHENA Research Center, Patras Science Park, Greece(工业系统研究所,ATHENA研究中心,帕特拉斯科学公园,希腊);AviSense.AI, Patras Science Park, Greece(AviSense.AI,帕特拉斯科学公园,希腊);Information Technologies Institute, CERTH, Greece(信息科技研究所,CERTH,希腊);KIOS Research and Innovation Center of Excellence, University of Cyprus, Cyprus(KIOS研究与创新中心,塞浦路斯大学,塞浦路斯)
MTD: Multi-Timestep Detector for Delayed Streaming Perception
School of Computer and Electronic Information, Guangxi University(广西大学计算机与电子信息学院);Guangxi Intelligent Digital Services Research Center of Engineering Technology(广西智能数字服务工程技术研究中心)
Optimizing Indoor Farm Monitoring Efficiency Using UAV: Yield Estimation in a GNSS-Denied Cherry Tomato Greenhouse
Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST)(机械工程系,乌山国立科学技术研究院);Agricultural Robotics and Artificial Intelligence, Metafarmers(农业机器人与人工智能,Metafarmers)
Smart Roads: Roadside Perception, Vehicle-Road Cooperation and Business Model
Xidian University(西安电子科技大学);Guangzhou Institute of Technology, Xidian University(西安电子科技大学广州研究院);Eindhoven University of Technology(埃因霍温理工大学);Nanyang Technological University(南洋理工大学);University of Oslo(奥斯陆大学);Simula Metropolitan Center for Digital Engineering(西穆拉大都会数字工程中心)
Intermittent Deployment for Large-Scale Multi-Robot Forage Perception: Data Synthesis, Prediction, and Planning
Virginia Tech(弗吉尼亚理工大学);University of Illinois, Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);Oregon State University(俄勒冈州立大学);University of Maryland, College Park(马里兰大学帕克分校)
Scalable Radar-based Roadside Perception: Self-localization and Occupancy Heat Map for Traffic Analysis
Fraunhofer IVI(弗劳恩霍夫应用信息学研究所(IVI));Karlsruhe Institute of Technology (KIT)(卡尔斯鲁厄理工学院);Technische Hochschule Ingolstadt(英戈尔施塔特应用技术大学);Technical University of Munich(慕尼黑工业大学);Fraunhofer IOSB(弗劳恩霍夫光学、电子和图像利用研究所(IOSB))