The increasing connectivity and intricate remote access environment have made traditional perimeter-based network defense vulnerable. Zero trust becomes a promising approach to provide defense policies based on agent-centric trust evaluation. However, the limited observations of the agent's trace bring information asymmetry in the decision-making. To facilitate the human understanding of the policy and the technology adoption, one needs to create a zero-trust defense that is explainable to humans and adaptable to different attack scenarios. To this end, we propose a scenario-agnostic zero-trust defense based on Partially Observable Markov Decision Processes (POMDP) and first-order Meta-Learning using only a handful of sample scenarios. The framework leads to an explainable and generalizable trust-threshold defense policy. To address the distribution shift between empirical security datasets and reality, we extend the model to a robust zero-trust defense minimizing the worst-case loss. We use case studies and real-world attacks to corroborate the results.
We consider the problem of providing users of deep Reinforcement Learning (RL) based systems with a better understanding of when their output can be trusted. We offer an explainable artificial intelligence (XAI) framework that provides a three-fold explanation: a graphical depiction of the systems generalization and performance in the current game state, how well the agent would play in semantically similar environments, and a narrative explanation of what the graphical information implies. We created a user-interface for our XAI framework and evaluated its efficacy via a human-user experiment…
Trustworthy and Explainable Deep Reinforcement Learning for Safe and Energy-Efficient Process Control: A Use Case in Industrial Compressed Air Systems
Fraunhofer Institute for Manufacturing Engineering and Automation IPA（弗劳恩霍夫制造工程与自动化研究所）；Institute for Energy Efficiency in Production EEP, University of Stuttgart（生产能效研究所，斯图加特大学）；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（工业制造与管理研究所，斯图加特大学）
University of Malaya（马来亚大学）；University of Hamburg（汉堡大学）
Recent applications of autonomous agents and robots, such as self-driving cars, scenario-based trainers, exploration robots, and service robots have brought attention to crucial trust-related challenges associated with the current generation of artificial intelligence (AI) systems. AI systems based on the connectionist deep learning neural network approach lack capabilities of explaining their decisions and actions to others, despite their great successes. Without symbolic interpretation capabilities, they are black boxes, which renders their decisions or actions opaque, making it difficult to…
The rapid growth of research in explainable artificial intelligence (XAI) follows on two substantial developments. First, the enormous application success of modern machine learning methods, especially deep and reinforcement learning, which have created high expectations for industrial, commercial and social value. Second, the emergence of concern for creating trusted AI systems, including the creation of regulatory principles to ensure transparency and trust of AI systems.These two threads have created a kind of "perfect storm" of research activity, all eager to create and deliver it any set …
A Graph-Enhanced Defense Framework for Explainable Fake News Detection with LLM
School of Artificial Intelligence, Jilin University（吉林大学人工智能学院）；Hong Kong Baptist University（香港浸会大学）；Guangdong Institute of Smart Education, Jinan University（暨南大学广东智慧教育研究院）；University of Science and Technology Beijing（北京科技大学）；Engineering Research Center of Knowledge-Driven Human-Machine Intelligence（知识驱动人机智能教育部工程研究中心）；International Center of Future Science, Jilin University（吉林大学未来科学国际合作中心）
Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations
Research Center Trustworthy Data Science and Security（可信数据科学与安全研究所以）；TU Dortmund University（图卢兹大学）；University Duisburg-Essen（杜伊斯堡- Essen大学）；University of Nottingham（诺丁汉大学）
Explainable Artificial Intelligence (XAI) techniques are frequently required by users in many AI systems with the goal of understanding complex models, their associated predictions, and gaining trust. While suitable for some specific tasks during development, their adoption by organisations to enhance trust in machine learning systems has unintended consequences. In this paper we discuss XAI's limitations in deployment and conclude that transparency alongside with rigorous validation are better suited to gaining trust in AI systems.
Universiti Putra Malaysia（马来西亚博特拉大学）；University of Salford（索尔福德大学）
This paper presents a trust computation for international relations and its calculus, which related to Bayesian inference, Dempster Shafer theory and subjective logic. We proposed a method that allows a trust computation which is previously subjective and incomputable. An example of case study for the trust computation is the United States of America Great Britain relations. The method supports decision makers in a government such as foreign ministry, defense ministry, presidential or prime minister office. The Department of Defense (DoD) may use our method to determine a nation that can be kn…
US Army Futures Command（美国陆军未来司令部）；US Army Engineer Research and Development Center（美国陆军工程研究与发展中心）；University of Massachusetts（马萨诸塞大学）
To benefit from AI advances, users and operators of AI systems must have reason to trust it. Trust arises from multiple interactions, where predictable and desirable behavior is reinforced over time. Providing the system's users with some understanding of AI operations can support predictability, but forcing AI to explain itself risks constraining AI capabilities to only those reconcilable with human cognition. We argue that AI systems should be designed with features that build trust by bringing decision-analytic perspectives and formal tools into AI. Instead of trying to achieve explainable …
The ability to explain the prediction of deep learning models to end-users is an important feature to leverage the power of artificial intelligence (AI) for the medical decision-making process, which is usually considered non-transparent and challenging to comprehend. In this paper, we apply state-of-the-art eXplainable artificial intelligence (XAI) methods to explain the prediction of the black-box AI models in the thyroid nodule diagnosis application. We propose new statistic-based XAI methods, namely Kernel Density Estimation and Density map, to explain the case of no nodule detected. XAI m…
TrustGuard: GNN-based Robust and Explainable Trust Evaluation with Dynamicity Support
Xidian University（西安电子科技大学）；Purdue University（普渡大学）；University of Alberta（阿尔伯塔大学）；Polish Academy of Sciences（波兰科学院）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）；Istinye University（伊斯蒂涅大学）
Building Trust in Conversational AI: A Comprehensive Review and Solution Architecture for Explainable, Privacy-Aware Systems using LLMs and Knowledge Graph
CeADAR - Ireland’s Centre for Applied AI, University College Dublin（爱尔兰应用人工智能中心CeADAR、都柏林大学学院）
Safety Instincts: LLMs Learn to Trust Their Internal Compass for Self-Defense
Beijing Institute of AI Safety and Governance（北京人工智能安全与治理研究院）；Beijing Key Laboratory of Safe AI and Superalignment（北京安全人工智能与超对齐重点实验室）；BrainCog Lab, Institute of Automation, Chinese Academy of Sciences（脑认知实验室，中国科学院自动化研究所）
Rutgers University, School of Arts and Sciences Newark（罗格斯大学纽瓦克文理学院）
Limited expert time is a key bottleneck in medical imaging. Due to advances in image classification, AI can now serve as decision-support for medical experts, with the potential for great gains in radiologist productivity and, by extension, public health. However, these gains are contingent on building and maintaining experts' trust in the AI agents. Explainable AI may build such trust by helping medical experts to understand the AI decision processes behind diagnostic judgements. Here we introduce and evaluate explanations based on Bayesian Teaching, a formal account of explanation rooted in …
University of South Carolina（南卡罗来纳大学）；Embible, Inc.（Embible公司）
AI systems have seen significant adoption in various domains. At the same time, further adoption in some domains is hindered by inability to fully trust an AI system that it will not harm a human. Besides the concerns for fairness, privacy, transparency, and explainability are key to developing trusts in AI systems. As stated in describing trustworthy AI "Trust comes through understanding. How AI-led decisions are made and what determining factors were included are crucial to understand." The subarea of explaining AI systems has come to be known as XAI. Multiple aspects of an AI system can be …
Promoting Security and Trust on Social Networks: Explainable Cyberbullying Detection Using Large Language Models in a Stream-Based Machine Learning Framework
Information Technologies Group - atlanTTic University of Vigo（信息科技组 - atlanTTic大学维戈分校）
Artificial intelligence (AI) enables machines to learn from human experience, adjust to new inputs, and perform human-like tasks. AI is progressing rapidly and is transforming the way businesses operate, from process automation to cognitive augmentation of tasks and intelligent process/data analytics. However, the main challenge for human users would be to understand and appropriately trust the result of AI algorithms and methods. In this paper, to address this challenge, we study and analyze the recent work done in Explainable Artificial Intelligence (XAI) methods and tools. We introduce a no…
A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language
Andalusian Research Institute in Data Science and Computational Intelligence（数据科学与计算智能安达卢西亚研究机构）；Universidad de Granada（格拉纳达大学）；Department of Computer Science（计算机科学系）；Artificial Intelligence Lab（人工智能实验室）；Vrije Universiteit Brussel（布鲁塞尔自由大学）
BACON: A fully explainable AI model with graded logic for decision making problems
Department of Information Systems University of Maryland, Baltimore County (UMBC)（信息系统系大学马里兰大学巴尔的摩县（UMBC））；Department of Computer Science San Francisco State University（计算机科学系旧金山州立大学）；Department of Information Systems UMBC（信息系统系大学马里兰大学巴尔的摩县（UMBC））
Explainable Artificial Intelligence (XAI) is a rising field in AI. It aims to produce a demonstrative factor of trust, which for human subjects is achieved through communicative means, which Machine Learning (ML) algorithms cannot solely produce, illustrating the necessity of an extra layer producing support to the model output. When approaching the medical field, we can see challenges arise when dealing with the involvement of human-subjects, the ideology behind trusting a machine to tend towards the livelihood of a human poses an ethical conundrum - leaving trust as the basis of the human-ex…
Tell Me Why: Designing an Explainable LLM-based Dialogue System for Student Problem Behavior Diagnosis
School of Educational Technology, Beijing Normal University, Beijing, China（北京师范大学教育技术学院）；Advanced Innovation Center for Future Education, Beijing Normal University, Beijing, China（北京师范大学未来教育创新中心）；Faculty of Education, The University of Hong Kong, Hong Kong SAR, China（香港大学教育学院）
