Autonomous systems with cognitive features are on their way into the market. Within complex environments, they promise to implement complex and goal oriented behavior even in a safety related context. This behavior is based on a certain level of situational awareness (perception) and advanced de-cision making (deliberation). These systems in many cases are driven by artificial intelligence (e.g. neural networks). The problem with such complex systems and with using AI technology is that there is no generally accepted approach to ensure trustworthiness. This paper presents a framework to exactly fill this gap. It proposes a reference lifecycle as a structured approach that is based on current safety standards and enhanced to meet the requirements of autonomous/cog-nitive systems and trustworthiness.
Loughborough University（拉夫堡大学）；University of Manchester（曼彻斯特大学）；Ulster University（阿尔斯特大学）
The trustworthiness of Robots and Autonomous Systems (RAS) has gained a prominent position on many research agendas towards fully autonomous systems. This research systematically explores, for the first time, the key facets of human-centered AI (HAI) for trustworthy RAS. In this article, five key properties of a trustworthy RAS initially have been identified. RAS must be (i) safe in any uncertain and dynamic surrounding environments; (ii) secure, thus protecting itself from any cyber-threats; (iii) healthy with fault tolerance; (iv) trusted and easy to use to allow effective human-machine inte…
The University of Sheffield（谢菲尔德大学）；University of Southampton（南安普顿大学）
Recent advances in artificial intelligence, specifically machine learning, contributed positively to enhancing the autonomous systems industry, along with introducing social, technical, legal and ethical challenges to make them trustworthy. Although Trustworthy Autonomous Systems (TAS) is an established and growing research direction that has been discussed in multiple disciplines, e.g., Artificial Intelligence, Human-Computer Interaction, Law, and Psychology. The impact of TAS on education curricula and required skills for future TAS engineers has rarely been discussed in the literature. This…
CT-SAFR: Safe and Interpretable Chain-of-Thought Reasoning for Autonomous Robots: A Multi-Layered Verification Framework for Trustworthy AI-Driven Robotic Decision Making
Trusted Autonomous Systems, Defence Cooperative Research Centre（可信自主系统国防合作研究中心）；Athena AI（雅典娜人工智能公司）；The University of Queensland（昆士兰大学）
What does it mean to be responsible and responsive when developing and deploying trusted autonomous systems in Defence? In this short reflective article, we describe a case study of building a trusted autonomous system - Athena AI - within an industry-led, government-funded project with diverse collaborators and stakeholders. Using this case study, we draw out lessons on the value and impact of embedding responsible research and innovation-aligned, ethics-by-design approaches and principles throughout the development of technology at high translation readiness levels.
Explainable AI for Safe and Trustworthy Autonomous Driving: A Systematic Review
School of Informatics, University of Edinburgh（爱丁堡大学信息学院）；Institute of Automotive Engineering, Technical University (TU) of Darmstadt（达姆施塔特工业大学汽车工程研究所）；School of Engineering and Physical Sciences, Heriot-Watt University（赫瑞瓦特大学工程与物理科学学院）
Universitat Politècnica de Catalunya - BarcelonaTech（加泰罗尼亚理工大学）；Fraunhofer IESE（弗劳恩霍夫实验软件工程研究所）
Companies dealing with Artificial Intelligence (AI) models in Autonomous Systems (AS) face several problems, such as users' lack of trust in adverse or unknown conditions, gaps between software engineering and AI model development, and operation in a continuously changing operational environment. This work-in-progress paper aims to close the gap between the development and operation of trustworthy AI-based AS by defining an approach that coordinates both activities. We synthesize the main challenges of AI-based AS in industrial settings. We reflect on the research efforts required to overcome …
One of the main problems of modern cognitive architectures is an excessively schematic approach to modeling the processes of cognitive activity. It does not allow the creation of a universal architecture that would be capable of reproducing mental functions without using a predetermined set of perceptual patterns. This paper considers an evolutionary approach to creating a cognitive functionality. The basis of our approach is the use of the functional kernel which consistently generates the intellectual functions of an autonomous agent. We consider a cognitive architecture which ensures the ev…
In this paper, we describe an integrated framework for autonomous decision making in a dynamic and interactive environment. We model the interactions between the ego agent and its operating environment as a two-player dynamic game, and integrate cognitive behavioral models, Bayesian inference, and receding-horizon optimal control to define a dynamically-evolving decision strategy for the ego agent. Simulation examples representing autonomous vehicle control in three traffic scenarios where the autonomous ego vehicle interacts with a human-driven vehicle are reported.
A new generation of increasingly autonomous and self-learning embodied systems is about to be developed. When deploying embodied systems into a real-life context we face various engineering challenges, as it is crucial to coordinate the behavior of embodied systems in a beneficial manner, ensure their compatibility with our human-centered social values, and design verifiably safe and reliable human-machine interaction. We are arguing that traditional systems engineering is coming to a climacteric from embedded to embodied systems, and with assuring the trustworthiness of dynamic federations of…
University of Pisa（比萨大学）；Information Technology for Market Leadership（市场领导力信息技术公司）；Institute of Information Science and Technologies (ISTI), CNR（意大利国家研究委员会信息科学与技术研究所）；AVL List GmbH（AVL李斯特有限公司）；Graz University of Technology（格拉茨工业大学）；Harokopio University of Athens（雅典哈罗科皮奥大学）；Infineon Technologies AG（英飞凌科技股份公司）；Marelli Europe S.p.A（玛涅蒂玛瑞利欧洲股份公司）
This paper discusses the perspective of the H2020 TEACHING project on the next generation of autonomous applications running in a distributed and highly heterogeneous environment comprising both virtual and physical resources spanning the edge-cloud continuum. TEACHING puts forward a human-centred vision leveraging the physiological, emotional, and cognitive state of the users as a driver for the adaptation and optimization of the autonomous applications. It does so by building a distributed, embedded and federated learning system complemented by methods and tools to enforce its dependability,…
University of Calgary（卡尔加里大学）；Óbuda University（奥布达大学）；Simon Fraser University（西蒙菲莎大学）；University of Waterloo（滑铁卢大学）；Bogazici University（海峡大学）；Systems Research Institute, Polish Academy of Sciences（波兰科学院系统研究所）；City University of Hong Kong（香港城市大学）；New Jersey Institute of Technology（新泽西理工学院）；University of Toronto（多伦多大学）；Furaxa, Inc.（Furaxa公司）；South China University of Technology（华南理工大学）；De Montfort University（德蒙福特大学）；Toronto Research Centre, DRDC（加拿大国防研究与发展部多伦多研究中心）；Raytheon Technologies Research Center（雷神技术研究中心）
Symbiotic Autonomous Systems (SAS) are advanced intelligent and cognitive systems exhibiting autonomous collective intelligence enabled by coherent symbiosis of human-machine interactions in hybrid societies. Basic research in the emerging field of SAS has triggered advanced general AI technologies functioning without human intervention or hybrid symbiotic systems synergizing humans and intelligent machines into coherent cognitive systems. This work presents a theoretical framework of SAS underpinned by the latest advances in intelligence, cognition, computer, and system sciences. SAS are char…
The potential market for modern self-driving cars is enormous, as they are developing remarkably rapidly. At the same time, however, accidents of pedestrian fatalities caused by autonomous driving have been recorded in the case of street crossing. To ensure traffic safety in self-driving environments and respond to vehicle-human interaction challenges such as jaywalking, we propose Level-$k$ Meta Reinforcement Learning (LK-MRL) algorithm. It takes into account the cognitive hierarchy of pedestrian responses and enables self-driving vehicles to adapt to various human behaviors. %which takes int…
CHARMS: A Cognitive Hierarchical Agent for Reasoning and Motion Stylization in Autonomous Driving
School of Mechanical and Electronic Engineering, Wuhan University of Technology（武汉理工大学机械与电子工程学院）；Intelligent Transportation Systems Research Center, Wuhan University of Technology（武汉理工大学智能交通系统研究所）；Department of Data and Systems Engineering, The University of Hong Kong（香港大学数据与系统工程系）；School of Computer Science and Artificial Intelligence, Wuhan University of Technology（武汉理工大学计算机科学与人工智能学院）；School of Mechanical Engineering, Southeast University（东南大学机械工程学院）
This research proposal aims to use cognitive methods to analyze the quality of roads based on the new proposed technology called Cognitive Internet of Vehicles (CIoV). By using Big Data corresponding to the collected data of autonomous vehicles, we can apply cognitive analytics to a huge amount of transportation data. This process can help us to create valuable information such as road quality from an immense volume of meaningless data. In this proposal, we are going to focus on the quality of roads for various business and commercial purposes. The proposed system can be used as an additional …
Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University（多媒体信息处理国家重点实验室，计算机学院，北京大学）；The University of Hong Kong（香港大学）
What Do They See? Interpreting Complex Road Scenarios Through the Eyes of Vision-Language-Action Models for Safe and Trustworthy Autonomous Vehicle Learning
Birla Institute of Technology and Science, Pilani（贝拉理工科学学院皮拉尼分校）；University of California, Merced（加州大学默塞德分校）
SAFEFLOW: A Principled Protocol for Trustworthy and Transactional Autonomous Agent Systems
Texas A&M University（德州农工大学）；UC San Diego（加州大学圣地亚哥分校）；UC Irvine（加州大学欧文分校）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Carnegie Mellon University（卡内基梅隆大学）；University of Michigan（密歇根大学）；Columbia University（哥伦比亚大学）；Meta
AIVV: Neuro-Symbolic LLM Agent-Integrated Verification and Validation for Trustworthy Autonomous Systems
School of Mechanical Engineering, Purdue University（普渡大学机械工程学院）；School of Electrical and Computer Engineering, Purdue University（普渡大学电气与计算机工程学院）；Department of Computer Science, Purdue University（普渡大学计算机科学系）；Department of Mathematics, Purdue University（普渡大学数学系）
Collaborative Trustworthiness for Good Decision Making in Autonomous Systems
Institute of Computer and Network Engineering (IDA) Technische Universität Braunschweig（计算机与网络工程学院（IDA） 拜鲁恩堡大学）；Chair of Embedded Systems TU Dortmund University（嵌入式系统教授 鄂尔敦大学）；Robert Bosch Corporate Research（博世企业研究）
Cognitive Platform Engineering for Autonomous Cloud Operations
IEEE Senior Texas USA（IEEE高级工程师，得克萨斯州，美国）；Albertsons Companies California USA（Albertsons公司，加利福尼亚州，美国）；Cleveland State University Ohio USA（克利夫兰州立大学，俄亥俄州，美国）；IEEE Senior Massachusetts USA（IEEE高级工程师，马萨诸塞州，美国）；IEEE Senior California USA（IEEE高级工程师，加利福尼亚州，美国）
