Explainable AI-based Intrusion Detection System for Industry 5.0: An Overview of the Literature, associated Challenges, the existing Solutions, and Potential Research Directions
Hamad bin Khalifa University（哈马德·本·哈利法大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；Munster Technological University（芒斯特理工大学）；Toronto Metropolitan University（多伦多都会大学）
Naseem Khan, Kashif Ahmad, Aref Al Tamimi, Mohammed M. Alani, Amine Bermak, Issa Khalil
Industry 5.0, which focuses on human and Artificial Intelligence (AI) collaboration for performing different tasks in manufacturing, involves a higher number of robots, Internet of Things (IoTs) devices and interconnections, Augmented/Virtual Reality (AR), and other smart devices. The huge involvement of these devices and interconnection in various critical areas, such as economy, health, education and defense systems, poses several types of potential security flaws. AI itself has been proven a very effective and powerful tool in different areas of cybersecurity, such as intrusion detection, malware detection, and phishing detection, among others. Just as in many application areas, cybersecurity professionals were reluctant to accept black-box ML solutions for cybersecurity applications. This reluctance pushed forward the adoption of eXplainable Artificial Intelligence (XAI) as a tool that helps explain how decisions are made in ML-based systems. In this survey, we present a comprehensive study of different XAI-based intrusion detection systems for industry 5.0, and we also examine the impact of explainability and interpretability on Cybersecurity practices through the lens of Adversarial XIDS (Adv-XIDS) approaches. Furthermore, we analyze the possible opportunities and challenges in XAI cybersecurity systems for industry 5.0 that elicit future research toward XAI-based solutions to be adopted by high-stakes industry 5.0 applications. We believe this rigorous analysis will establish a foundational framework for subsequent research endeavors within the specified domain.
Cybersecurity is a domain where the data distribution is constantly changing with attackers exploring newer patterns to attack cyber infrastructure. Intrusion detection system is one of the important layers in cyber safety in today's world. Machine learning based network intrusion detection systems started showing effective results in recent years. With deep learning models, detection rates of network intrusion detection system are improved. More accurate the model, more the complexity and hence less the interpretability. Deep neural networks are complex and hard to interpret which makes diffi…
Mississippi State University（密西西比州立大学）；U.S Army Engineer Research and Development Center（美国陆军工程研究与发展中心）
Modern Artificial Intelligence (AI) enabled Intrusion Detection Systems (IDS) are complex black boxes. This means that a security analyst will have little to no explanation or clarification on why an IDS model made a particular prediction. A potential solution to this problem is to research and develop Explainable Intrusion Detection Systems (X-IDS) based on current capabilities in Explainable Artificial Intelligence (XAI). In this paper, we create a Self Organizing Maps (SOMs) based X-IDS system that is capable of producing explanatory visualizations. We leverage SOM's explainability to creat…
ExAI5G: A Logic-Based Explainable AI Framework for Intrusion Detection in 5G Networks
organization= Faculty of Information Technology；Electrical Engineering, University of Oulu , addressline= , city= Oulu , postcode= 90570 , state= , country= Finland
Explainable AI for Comparative Analysis of Intrusion Detection Models
Embry-Riddle Aeronautical University（安柏瑞德航空大学）；University of Tennessee at Martin（田纳西大学马丁分校）；Bowling Green State University（鲍灵格林州立大学）；University of Maryland, Baltimore County（马里兰大学巴尔的摩郡分校）
Mississippi State University（密西西比州立大学）；U.S Army Engineer Research and Development Center（美国陆军工程研究与发展中心）
The application of Artificial Intelligence (AI) and Machine Learning (ML) to cybersecurity challenges has gained traction in industry and academia, partially as a result of widespread malware attacks on critical systems such as cloud infrastructures and government institutions. Intrusion Detection Systems (IDS), using some forms of AI, have received widespread adoption due to their ability to handle vast amounts of data with a high prediction accuracy. These systems are hosted in the organizational Cyber Security Operation Center (CSoC) as a defense tool to monitor and detect malicious network…
XAI-SOH-FL: Enhancing SOH-FL with Adaptive Aggregation and Explainable AI for Intrusion Detection in Heterogeneous IoT
School of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST)（电气工程与计算机科学学院（SEECS），国家 sciences and Technology（NUST））
Explainable AI-Driven Cyber Risk Analytics and Model Reliability Assessment for Intelligent Governance of U.S. Critical Infrastructure: An XGBoost and SHAP-Based Intrusion Detection Framework
Department of Business Information Systems, Central Michigan University（中央密歇根大学商业信息系统系）；Department of Information Studies, Trine University（特林大学信息学系）；Department of Computer Science, Maharishi International University（Maharishi国际大学计算机科学系）
LENS-XAI: Redefining Lightweight and Explainable Network Security through Knowledge Distillation and Variational Autoencoders for Scalable Intrusion Detection in Cybersecurity
Kırıkkale University（克尔克卡拉大学）；University College Dublin（都柏林大学学院）；CeADAR；School of Computer Science（计算机科学学院）
University of Washington（华盛顿大学）；Arthur AI；BITS Pilani, Goa Campus（贝拉理工学院果阿校区）；University of Maryland（马里兰大学）
As machine learning (ML) systems take a more prominent and central role in contributing to life-impacting decisions, ensuring their trustworthiness and accountability is of utmost importance. Explanations sit at the core of these desirable attributes of a ML system. The emerging field is frequently called ``Explainable AI (XAI)'' or ``Explainable ML.'' The goal of explainable ML is to intuitively explain the predictions of a ML system, while adhering to the needs to various stakeholders. Many explanation techniques were developed with contributions from both academia and industry. However, the…
Conversational versus Dashboard Explainable AI for UAV Intrusion Detection: An Empirical Study of Operator Trust and Reliance
Phenikaa University（菲卡大学）；Phenikaa School of Computing（菲卡计算机学院）；MobiFone Corporation（MobiFone集团）；MobiFone HighTech Center（MobiFone高科技中心）；National Economics University（国民经济大学）；College of Technology（技术学院）
Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD), Porto School of Engineering (ISEP)（波尔图工程学院（ISEP）智能工程与先进创新开发计算研究组（GECAD））
Cybersecurity has been a concern for quite a while now. In the latest years, cyberattacks have been increasing in size and complexity, fueled by significant advances in technology. Nowadays, there is an unavoidable necessity of protecting systems and data crucial for business continuity. Hence, many intrusion detection systems have been created in an attempt to mitigate these threats and contribute to a timelier detection. This work proposes an interpretable and explainable hybrid intrusion detection system, which makes use of artificial intelligence methods to achieve better and more long-las…
Human-Centered Explainable AI for Security Enhancement: A Deep Intrusion Detection Framework
Department of Computer Science and Engineering, United International University (UIU), Dhaka 1212, Bangladesh（计算机科学与工程系，国际联合大学（UIU），达卡1212，孟加拉国）；Department of Electrical and Electronic Engineering, Islamic University of Technology, Gazipur 1704, Bangladesh（电气与电子工程系，伊斯兰科技大学，加兹ipur 1704，孟加拉国）；Department of Computer Science and Engineering (CSE), University of Asia Pacific (UAP), Dhaka 1207, Bangladesh（计算机科学与工程系（CSE），亚洲太平洋大学（UAP），达卡1207，孟加拉国）；Department of Information Systems, University of Maryland, Baltimore, 21250, Maryland, USA（信息系统系，马里兰大学，巴尔的摩，21250，美国）；College Of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802, USA（信息科学与技术学院，宾夕法尼亚州立大学，大学公园，PA 16802，美国）；Department of Information Technology, Washington University of Science and Technology, Alexandria, VA（信息技术系，科学与技术华盛顿大学，亚历山大，VA）；College of Engineering and Information Technology, University of Maryland, College Park, 20742, Maryland, USA（工程与信息技术学院，马里兰大学，学院公园，20742，美国）
School of Technology, Pandit Deendayal Energy University (PDEU)（潘迪特·迪恩达亚尔能源大学技术学院）；AI Institute, University of South Carolina（南卡罗来纳大学人工智能学院）
The objective of an online Mart is to match buyers and sellers, to weigh animals and to oversee their sale. A reliable pricing method can be developed by ML models that can read through historical sales data. However, when AI models suggest or recommend a price, that in itself does not reveal too much (i.e., it acts like a black box) about the qualities and the abilities of an animal. An interested buyer would like to know more about the salient features of an animal before making the right choice based on his requirements. A model capable of explaining the different factors that impact the pr…
Mississippi State University（密西西比州立大学）；U.S. Army Engineer Research and Development Center（美国陆军工程师研发中心）
The current state of the art systems in Artificial Intelligence (AI) enabled intrusion detection use a variety of black box methods. These black box methods are generally trained using Error Based Learning (EBL) techniques with a focus on creating accurate models. These models have high performative costs and are not easily explainable. A white box Competitive Learning (CL) based eXplainable Intrusion Detection System (X-IDS) offers a potential solution to these problem. CL models utilize an entirely different learning paradigm than EBL approaches. This different learning process makes the CL …
Artificial Intelligence (AI) has become an integral part of modern-day security solutions for its ability to learn very complex functions and handling "Big Data". However, the lack of explainability and interpretability of successful AI models is a key stumbling block when trust in a model's prediction is critical. This leads to human intervention, which in turn results in a delayed response or decision. While there have been major advancements in the speed and performance of AI-based intrusion detection systems, the response is still at human speed when it comes to explaining and interpreting…
Enhancing Adversarial Robustness of IoT Intrusion Detection via SHAP-Based Attribution Fingerprinting
School of Information Technology, York University（约克大学信息科技学院）；School of Computer Science, University of Guelph（圭尔夫大学计算机科学学院）；National Research Council of Canada（加拿大国家研究委员会）
With the advent of Industry 4.0, Data Science and Explainable Artificial Intelligence (XAI) has received considerable intrest in recent literature. However, the entry threshold into XAI, in terms of computer coding and the requisite mathematical apparatus, is really high. For fault diagnosis of steel plates, this work reports on a methodology of incorporating XAI based insights into the Data Science process of development of high precision classifier. Using Synthetic Minority Oversampling Technique (SMOTE) and notion of medoids, insights from XAI tools viz. Ceteris Peribus profiles, Partial De…
Lack of transparency has been the Achilles heal of Neural Networks and their wider adoption in industry. Despite significant interest this shortcoming has not been adequately addressed. This study proposes a novel framework called Hide-and-Seek (HnS) for training Interpretable Neural Networks and establishes a theoretical foundation for exploring and comparing similar ideas. Extensive experimentation indicates that a high degree of interpretability can be imputed into Neural Networks, without sacrificing their predictive power.
University of Hartford（哈特福德大学）；Tennessee Tech University（田纳西理工大学）；Edith Cowan University（伊迪丝·考恩大学）
The lack of explainability of a decision from an Artificial Intelligence (AI) based "black box" system/model, despite its superiority in many real-world applications, is a key stumbling block for adopting AI in many high stakes applications of different domain or industry. While many popular Explainable Artificial Intelligence (XAI) methods or approaches are available to facilitate a human-friendly explanation of the decision, each has its own merits and demerits, with a plethora of open challenges. We demonstrate popular XAI methods with a mutual case study/task (i.e., credit default predicti…
Playtech plc（Playtech公司）；City, University of London（伦敦大学城市学院）
Recent AI-related scandals have shed a spotlight on accountability in AI, with increasing public interest and concern. This paper draws on literature from public policy and governance to make two contributions. First, we propose an AI accountability ecosystem as a useful lens on the system, with different stakeholders requiring and contributing to specific accountability mechanisms. We argue that the present ecosystem is unbalanced, with a need for improved transparency via AI explainability and adequate documentation and process formalisation to support internal audit, leading up eventually t…
The concept of Industry 4.0 brings a disruption into the processing industry. It is characterised by a high degree of intercommunication, embedded computation, resulting in a decentralised and distributed handling of data. Additionally, cloud-storage and Software-as-a-Service (SaaS) approaches enhance a centralised storage and handling of data. This often takes place in third-party networks. Furthermore, Industry 4.0 is driven by novel business cases. Lot sizes of one, customer individual production, observation of process state and progress in real-time and remote maintenance, just to name a …
Automated and Explainable Denial of Service Analysis for AI-Driven Intrusion Detection Systems
Computer and Information Science Department, Fordham University（福特汉姆大学计算机与信息科学系）；RENCI, University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校RENCI）；Department of Mathematics and Computer Science, Royal Military College of Canada（加拿大皇家军事学院数学与计算机科学系）；Data Science Department, University of North Texas（德克萨斯大学纳夫卡特数据科学系）
Pandit Deendayal Petroleum University（潘迪特·迪恩达亚尔石油大学）；Confirm SFI Centre for Smart Manufacturing（Confirm SFI智能制造中心）；Data Science Institute（数据科学研究所）；National University of Ireland Galway（爱尔兰国立大学戈尔韦分校）
With the increasing availability of structured and unstructured data and the swift progress of analytical techniques, Artificial Intelligence (AI) is bringing a revolution to the healthcare industry. With the increasingly indispensable role of AI in healthcare, there are growing concerns over the lack of transparency and explainability in addition to potential bias encountered by predictions of the model. This is where Explainable Artificial Intelligence (XAI) comes into the picture. XAI increases the trust placed in an AI system by medical practitioners as well as AI researchers, and thus, ev…
University College Dublin（都柏林大学学院）；Technological University Dublin（都柏林理工大学）；Ho Chi Minh City University of Technology and Education（胡志明市工业大学）；Vellore Institute of Technology（韦洛尔理工学院）；Lebanese American University（黎巴嫩美国大学）
When 5G began its commercialisation journey around 2020, the discussion on the vision of 6G also surfaced. Researchers expect 6G to have higher bandwidth, coverage, reliability, energy efficiency, lower latency, and an integrated "human-centric" network system powered by artificial intelligence (AI). Such a 6G network will lead to an excessive number of automated decisions made in real-time. These decisions can range widely, from network resource allocation to collision avoidance for self-driving cars. However, the risk of losing control over decision-making may increase due to high-speed, dat…
Despite the growing popularity of modern machine learning techniques (e.g. Deep Neural Networks) in cyber-security applications, most of these models are perceived as a black-box for the user. Adversarial machine learning offers an approach to increase our understanding of these models. In this paper we present an approach to generate explanations for incorrect classifications made by data-driven Intrusion Detection Systems (IDSs). An adversarial approach is used to find the minimum modifications (of the input features) required to correctly classify a given set of misclassified samples. The m…
Evaluating RE Practices for Explainability: Synthesizing Insights from Daimler Truck into an Explainable RE Framework Proposal
TUM School of CIT, Technical University of Munich（慕尼黑技术大学 CIT 学院）；Institute of Software Engineering, University of Stuttgart（斯图加特大学软件工程研究所）；Department of Computer Science, Vrije Universiteit Amsterdam（阿姆斯特丹自由大学计算机科学系）
