Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework
Department of Computer Science and Engineering, United International University (UIU), Dhaka 1212, Bangladesh(计算机科学与工程系,国际联合大学(UIU),达卡1212,孟加拉国);Department of Electrical and Electronic Engineering, Islamic University of Technology, Gazipur 1704, Bangladesh(电气与电子工程系,伊斯兰科技大学,加兹ipur 1704,孟加拉国);Department of Computer Science and Engineering (CSE), University of Asia Pacific (UAP), Dhaka 1207, Bangladesh(计算机科学与工程系(CSE),亚洲太平洋大学(UAP),达卡1207,孟加拉国);Department of Information Systems, University of Maryland, Baltimore, 21250, Maryland, USA(信息系统系,马里兰大学,巴尔的摩,21250,美国);College Of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802, USA(信息科学与技术学院,宾夕法尼亚州立大学,大学公园,PA 16802,美国);Department of Information Technology, Washington University of Science and Technology, Alexandria, VA(信息技术系,科学与技术华盛顿大学,亚历山大,VA);College of Engineering and Information Technology, University of Maryland, College Park, 20742, Maryland, USA(工程与信息技术学院,马里兰大学,学院公园,20742,美国)
The increasing complexity and frequency of cyber-threats demand intrusion detection systems (IDS) that are not only accurate but also interpretable. This paper presented a novel IDS framework that integrated Explainable Artificial Intelligence (XAI) to enhance transparency in deep learning models. The framework was evaluated experimentally using the benchmark dataset NSL-KDD, demonstrating superior performance compared to traditional IDS and black-box deep learning models. The proposed approach combined Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) networks for capturing temporal dependencies in traffic sequences. Our deep learning results showed that both CNN and LSTM reached 0.99 for accuracy, whereas LSTM outperformed CNN at macro average precision, recall, and F-1 score. For weighted average precision, recall, and F-1 score, both models scored almost similarly. To ensure interpretability, the XAI model SHapley Additive exPlanations (SHAP) was incorporated, enabling security analysts to understand and validate model decisions. Some notable influential features were srv_serror_rate, dst_host_srv_serror_rate, and serror_rate for both models, as pointed out by SHAP. We also conducted a trust-focused expert survey based on IPIP6 and Big Five personality traits via an interactive UI to evaluate the system's reliability and usability. This work highlighted the potential of combining performance and transparency in cybersecurity solutions and recommends future enhancements through adaptive learning for real-time threat detection.
Mississippi State University(密西西比州立大学);U.S Army Engineer Research and Development Center(美国陆军工程研究与发展中心)
Modern Artificial Intelligence (AI) enabled Intrusion Detection Systems (IDS) are complex black boxes. This means that a security analyst will have little to no explanation or clarification on why an IDS model made a particular prediction. A potential solution to this problem is to research and develop Explainable Intrusion Detection Systems (X-IDS) based on current capabilities in Explainable Artificial Intelligence (XAI). In this paper, we create a Self Organizing Maps (SOMs) based X-IDS system that is capable of producing explanatory visualizations. We leverage SOM's explainability to creat…
Cybersecurity is a domain where the data distribution is constantly changing with attackers exploring newer patterns to attack cyber infrastructure. Intrusion detection system is one of the important layers in cyber safety in today's world. Machine learning based network intrusion detection systems started showing effective results in recent years. With deep learning models, detection rates of network intrusion detection system are improved. More accurate the model, more the complexity and hence less the interpretability. Deep neural networks are complex and hard to interpret which makes diffi…
ExAI5G: A Logic-Based Explainable AI Framework for Intrusion Detection in 5G Networks
organization= Faculty of Information Technology;Electrical Engineering, University of Oulu , addressline= , city= Oulu , postcode= 90570 , state= , country= Finland
Explainable AI for Comparative Analysis of Intrusion Detection Models
Embry-Riddle Aeronautical University(安柏瑞德航空大学);University of Tennessee at Martin(田纳西大学马丁分校);Bowling Green State University(鲍灵格林州立大学);University of Maryland, Baltimore County(马里兰大学巴尔的摩郡分校)
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)(电气工程与计算机科学学院(SEECS),国家 sciences and Technology(NUST))
Mississippi State University(密西西比州立大学);U.S Army Engineer Research and Development Center(美国陆军工程研究与发展中心)
The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network…
Explainable AI-based Intrusion Detection System for Industry 5.0: An Overview of the Literature, associated Challenges, the existing Solutions, and Potential Research Directions
Hamad bin Khalifa University(哈马德·本·哈利法大学);Qatar Computing Research Institute(卡塔尔计算研究所);Munster Technological University(芒斯特理工大学);Toronto Metropolitan University(多伦多都会大学)
In recent years, the field of explainable AI (XAI) has produced a vast collection of algorithms, providing a useful toolbox for researchers and practitioners to build XAI applications. With the rich application opportunities, explainability is believed to have moved beyond a demand by data scientists or researchers to comprehend the models they develop, to an essential requirement for people to trust and adopt AI deployed in numerous domains. However, explainability is an inherently human-centric property and the field is starting to embrace human-centered approaches. Human-computer interactio…
Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AI
Tübingen AI Center, University of Tübingen Germany(图宾根人工智能中心、图宾根大学德国);Fraunhofer Heinrich Hertz Institute Germany(弗劳恩霍夫海因里希·赫兹研究所德国);Information and Interaction Design, Yonsei University South Korea(信息与交互设计,延世大学韩国)
LENS-XAI: Redefining Lightweight and Explainable Network Security through Knowledge Distillation and Variational Autoencoders for Scalable Intrusion Detection in Cybersecurity
Kırıkkale University(克尔克卡拉大学);University College Dublin(都柏林大学学院);CeADAR;School of Computer Science(计算机科学学院)
Explanations--a form of post-hoc interpretability--play an instrumental role in making systems accessible as AI continues to proliferate complex and sensitive sociotechnical systems. In this paper, we introduce Human-centered Explainable AI (HCXAI) as an approach that puts the human at the center of technology design. It develops a holistic understanding of "who" the human is by considering the interplay of values, interpersonal dynamics, and the socially situated nature of AI systems. In particular, we advocate for a reflective sociotechnical approach. We illustrate HCXAI through a case study…
Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance
Phenikaa University(菲卡大学);Phenikaa School of Computing(菲卡计算机学院);MobiFone Corporation(MobiFone集团);MobiFone HighTech Center(MobiFone高科技中心);National Economics University(国民经济大学);College of Technology(技术学院)
Explainable AI-Driven Cyber Risk Analytics and Model Reliability Assessment for Intelligent Governance of U.S. Critical Infrastructure: An XGBoost and SHAP-Based Intrusion Detection Framework
Department of Business Information Systems, Central Michigan University(中央密歇根大学商业信息系统系);Department of Information Studies, Trine University(特林大学信息学系);Department of Computer Science, Maharishi International University(Maharishi国际大学计算机科学系)
Mind the XAI Gap: A Human-Centered LLM Framework for Democratizing Explainable AI
Aristotle University of Thessaloniki, School of Informatics(塞萨洛尼基亚里士多德大学信息学院);Technical University of Denmark(丹麦技术大学);Telefonica Scientific Research(电信科学研究院)
On the Design and Evaluation of Human-centered Explainable AI Systems: A Systematic Review and Taxonomy
Dresden University of Technology(德累斯顿技术大学);Charité Berlin(柏林夏里特医学院);Department Speculative Transformation(推测转化部门);Chair of Engineering Psychology and Applied Cognitive Research(工程心理学与应用认知研究教授职位)
Explainable AI-Based Interface System for Weather Forecasting Model
Korea Advanced Institute of Science and Technology(KAIST)(韩国科学技术院(KAIST));SI-Analytics;National Institute of Meteorological Sciences(NIMS)(气象厅气象研究所(NIMS));INEEJI
A surge of interest in explainable AI (XAI) has led to a vast collection of algorithmic work on the topic. While many recognize the necessity to incorporate explainability features in AI systems, how to address real-world user needs for understanding AI remains an open question. By interviewing 20 UX and design practitioners working on various AI products, we seek to identify gaps between the current XAI algorithmic work and practices to create explainable AI products. To do so, we develop an algorithm-informed XAI question bank in which user needs for explainability are represented as prototy…
In healthcare, AI systems support clinicians and patients in diagnosis, treatment, and monitoring, but many systems' poor explainability remains challenging for practical application. Overcoming this barrier is the goal of explainable AI (XAI). However, an explanation can be perceived differently and, thus, not solve the black-box problem for everyone. The domain of Human-Centered AI deals with this problem by adapting AI to users. We present a user-centered persona concept to evaluate XAI and use it to investigate end-users preferences for various explanation styles and contents in a mobile h…
Enhancing Adversarial Robustness of IoT Intrusion Detection via SHAP-Based Attribution Fingerprinting
School of Information Technology, York University(约克大学信息科技学院);School of Computer Science, University of Guelph(圭尔夫大学计算机科学学院);National Research Council of Canada(加拿大国家研究委员会)
Mississippi State University(密西西比州立大学);U.S. Army Engineer Research and Development Center(美国陆军工程师研发中心)
The current state of the art systems in Artificial Intelligence (AI) enabled intrusion detection use a variety of black box methods. These black box methods are generally trained using Error Based Learning (EBL) techniques with a focus on creating accurate models. These models have high performative costs and are not easily explainable. A white box Competitive Learning (CL) based eXplainable Intrusion Detection System (X-IDS) offers a potential solution to these problem. CL models utilize an entirely different learning paradigm than EBL approaches. This different learning process makes the CL …
University of Hartford(哈特福德大学);Tennessee Tech University(田纳西理工大学);Edith Cowan University(伊迪丝·考恩大学)
The lack of explainability of a decision from an Artificial Intelligence (AI) based "black box" system/model, despite its superiority in many real-world applications, is a key stumbling block for adopting AI in many high stakes applications of different domain or industry. While many popular Explainable Artificial Intelligence (XAI) methods or approaches are available to facilitate a human-friendly explanation of the decision, each has its own merits and demerits, with a plethora of open challenges. We demonstrate popular XAI methods with a mutual case study/task (i.e., credit default predicti…
Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD), Porto School of Engineering (ISEP)(波尔图工程学院(ISEP)智能工程与先进创新开发计算研究组(GECAD))
Cybersecurity has been a concern for quite a while now. In the latest years, cyberattacks have been increasing in size and complexity, fueled by significant advances in technology. Nowadays, there is an unavoidable necessity of protecting systems and data crucial for business continuity. Hence, many intrusion detection systems have been created in an attempt to mitigate these threats and contribute to a timelier detection. This work proposes an interpretable and explainable hybrid intrusion detection system, which makes use of artificial intelligence methods to achieve better and more long-las…
Institute for Informatics and Telematics (IIT), National Research Council of Italy (CNR)(意大利国家研究委员会(CNR)信息学与电信研究所(IIT))
Nowadays Artificial Intelligence (AI) has become a fundamental component of healthcare applications, both clinical and remote, but the best performing AI systems are often too complex to be self-explaining. Explainable AI (XAI) techniques are defined to unveil the reasoning behind the system's predictions and decisions, and they become even more critical when dealing with sensitive and personal health data. It is worth noting that XAI has not gathered the same attention across different research areas and data types, especially in healthcare. In particular, many clinical and remote health appl…
Non-technical end-users are silent and invisible users of the state-of-the-art explainable artificial intelligence (XAI) technologies. Their demands and requirements for AI explainability are not incorporated into the design and evaluation of XAI techniques, which are developed to explain the rationales of AI decisions to end-users and assist their critical decisions. This makes XAI techniques ineffective or even harmful in high-stakes applications, such as healthcare, criminal justice, finance, and autonomous driving systems. To systematically understand end-users' requirements to support the…
Automated and Explainable Denial of Service Analysis for AI-Driven Intrusion Detection Systems
Computer and Information Science Department, Fordham University(福特汉姆大学计算机与信息科学系);RENCI, University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校RENCI);Department of Mathematics and Computer Science, Royal Military College of Canada(加拿大皇家军事学院数学与计算机科学系);Data Science Department, University of North Texas(德克萨斯大学纳夫卡特数据科学系)
Artificial Intelligence (AI) has become an integral part of modern-day security solutions for its ability to learn very complex functions and handling "Big Data". However, the lack of explainability and interpretability of successful AI models is a key stumbling block when trust in a model's prediction is critical. This leads to human intervention, which in turn results in a delayed response or decision. While there have been major advancements in the speed and performance of AI-based intrusion detection systems, the response is still at human speed when it comes to explaining and interpreting…
Michigan Technological University(密歇根理工大学);Institute for Human and Machine Cognition(人类与机器认知研究所);Macrocognition, LLC(Macrocognition有限责任公司)
Explainable Artificial Intelligence (XAI) has re-emerged in response to the development of modern AI and ML systems. These systems are complex and sometimes biased, but they nevertheless make decisions that impact our lives. XAI systems are frequently algorithm-focused; starting and ending with an algorithm that implements a basic untested idea about explainability. These systems are often not tested to determine whether the algorithm helps users accomplish any goals, and so their explainability remains unproven. We propose an alternative: to start with human-focused principles for the design,…
Approximately 1.25 million people in the United States are treated each year for burn injuries. Precise burn injury classification is an important aspect of the medical AI field. In this work, we propose an explainable human-in-the-loop framework for improving burn ultrasound classification models. Our framework leverages an explanation system based on the LIME classification explainer to corroborate and integrate a burn expert's knowledge -- suggesting new features and ensuring the validity of the model. Using this framework, we discover that B-mode ultrasound classifiers can be enhanced by s…
A three-Level Framework for LLM-Enhanced eXplainable AI: From technical explanations to natural language
Andalusian Research Institute in Data Science and Computational Intelligence(数据科学与计算智能安达卢西亚研究机构);Universidad de Granada(格拉纳达大学);Department of Computer Science(计算机科学系);Artificial Intelligence Lab(人工智能实验室);Vrije Universiteit Brussel(布鲁塞尔自由大学)