Many current approaches to the design of intrusion detection systems apply feature selection in a static, non-adaptive fashion. These methods often neglect the dynamic nature of network data which requires to use adaptive feature selection techniques. In this paper, we present a simple technique based on incremental learning of support vector machines in order to rank the features in real time within a streaming model for network data. Some illustrative numerical experiments with two popular benchmark datasets show that our approach allows to adapt to the changes in normal network behaviour and novel attack patterns which have not been experienced before.
BARS: Benign-Anchored Ranking and Selection for False Alarm Reduction in Network Intrusion Detection
Department of Computer Science（计算机科学系）；New Mexico State University（新墨西哥州立大学）；Graduate School of Informatics（信息科学研究生院）；Osaka Metropolitan University（大阪 metropolitan 大学）
University of Western Ontario（西安大略大学）；University of Sharjah（沙迦大学）
Cyber-security garnered significant attention due to the increased dependency of individuals and organizations on the Internet and their concern about the security and privacy of their online activities. Several previous machine learning (ML)-based network intrusion detection systems (NIDSs) have been developed to protect against malicious online behavior. This paper proposes a novel multi-stage optimized ML-based NIDS framework that reduces computational complexity while maintaining its detection performance. This work studies the impact of oversampling techniques on the models' training samp…
University of Michigan（密歇根大学）；Facebook Infrastructure Engineering（脸书基础设施工程）；Microsoft（微软）
We present online boosting algorithms for multilabel ranking with top-k feedback, where the learner only receives information about the top k items from the ranking it provides. We propose a novel surrogate loss function and unbiased estimator, allowing weak learners to update themselves with limited information. Using these techniques we adapt full information multilabel ranking algorithms (Jung and Tewari, 2018) to the top-k feedback setting and provide theoretical performance bounds which closely match the bounds of their full information counterparts, with the cost of increased sample comp…
Given a set $V$ of $n$ objects, an online ranking system outputs at each time step a full ranking of the set, observes a feedback of some form and suffers a loss. We study the setting in which the (adversarial) feedback is an element in $V$, and the loss is the position (0th, 1st, 2nd...) of the item in the outputted ranking. More generally, we study a setting in which the feedback is a subset $U$ of at most $k$ elements in $V$, and the loss is the sum of the positions of those elements. We present an algorithm of expected regret $O(n^{3/2}\sqrt{Tk})$ over a time horizon of $T$ steps with resp…
nCMD: Benign-Anchored Feature Selection for Imbalanced Network Intrusion Detection
Department of Computer Science（计算机科学系）；New Mexico State University（新墨西哥州立大学）；Graduate School of Informatics（信息科学研究生院）；Osaka Metropolitan University（大阪 Metropolitan 大学）
National Technical University of Athens（雅典国立技术大学）；Singapore University of Technology and Design（新加坡科技设计大学）
We consider a natural model of online preference aggregation, where sets of preferred items $R_1, R_2, \ldots, R_t$ along with a demand for $k_t$ items in each $R_t$, appear online. Without prior knowledge of $(R_t, k_t)$, the learner maintains a ranking $π_t$ aiming that at least $k_t$ items from $R_t$ appear high in $π_t$. This is a fundamental problem in preference aggregation with applications to, e.g., ordering product or news items in web pages based on user scrolling and click patterns. The widely studied Generalized Min-Sum-Set-Cover (GMSSC) problem serves as a formal model for the set…
Existing online learning to rank (OL2R) solutions are limited to linear models, which are incompetent to capture possible non-linear relations between queries and documents. In this work, to unleash the power of representation learning in OL2R, we propose to directly learn a neural ranking model from users' implicit feedback (e.g., clicks) collected on the fly. We focus on RankNet and LambdaRank, due to their great empirical success and wide adoption in offline settings, and control the notorious explore-exploit trade-off based on the convergence analysis of neural networks using neural tangen…
Learning to rank is a supervised learning problem where the output space is the space of rankings but the supervision space is the space of relevance scores. We make theoretical contributions to the learning to rank problem both in the online and batch settings. First, we propose a perceptron-like algorithm for learning a ranking function in an online setting. Our algorithm is an extension of the classic perceptron algorithm for the classification problem. Second, in the setting of batch learning, we introduce a sufficient condition for convex ranking surrogates to ensure a generalization boun…
William & Mary（威廉与玛丽学院）；Old Dominion University（欧道明大学）
As the number of IoT devices has increased rapidly, IoT botnets have exploited the vulnerabilities of IoT devices. However, it is still challenging to detect the initial intrusion on IoT devices prior to massive attacks. Recent studies have utilized power side-channel information to identify this intrusion behavior on IoT devices but still lack accurate models in real-time for ubiquitous botnet detection. We proposed the first online intrusion detection system called DeepAuditor for IoT devices via power auditing. To develop the real-time system, we proposed a lightweight power auditing device…
In this paper, we consider the applications of process mining in intrusion detection. We propose a novel process mining inspired algorithm to be used to preprocess data in intrusion detection systems (IDS). The algorithm is designed to process the network packet data and it works well in online mode for online intrusion detection. To test our algorithm, we used the CSE-CIC-IDS2018 dataset which contains several common attacks. The packet data was preprocessed with this algorithm and then fed into the detectors. We report on the experiments using the algorithm with different machine learning (M…
In this paper, we propose an online learning algorithm PRIL for learning ranking classifiers using interval labeled data and show its correctness. We show its convergence in finite number of steps if there exists an ideal classifier such that the rank given by it for an example always lies in its label interval. We then generalize this mistake bound result for the general case. We also provide regret bound for the proposed algorithm. We propose a multiplicative update algorithm for PRIL called M-PRIL. We provide its correctness and convergence results. We show the effectiveness of PRIL by show…
Secure vehicular communication is a critical factor for secure traffic management. Effective security in intelligent transportation systems (ITS) requires effective and timely intrusion detection systems (IDS). In this paper, we consider false data injection attacks and distributed denial-of-service (DDoS) attacks, especially the stealthy DDoS attacks, targeting the integrity and availability, respectively, in vehicular ad-hoc networks (VANET). Novel statistical intrusion detection and mitigation techniques based on centralized communications through roadside units (RSU) are proposed for the c…
We propose a novel hierarchical online intrusion detection system (HOIDS) for supervisory control and data acquisition (SCADA) networks based on machine learning algorithms. By utilizing the server-client topology while keeping clients distributed for global protection, high detection rate is achieved with minimum network impact. We implement accurate models of normal-abnormal binary detection and multi-attack identification based on logistic regression and quasi-Newton optimization algorithm using the Broyden-Fletcher-Goldfarb-Shanno approach. The detection system is capable of accelerating d…
Open University of Cyprus（塞浦路斯开放大学）；University of Plymouth（普利茅斯大学）；University of Peloponnese（伯罗奔尼撒大学）
Attack vectors are continuously evolving in order to evade Intrusion Detection systems. Internet of Things (IoT) environments, while beneficial for the IT ecosystem, suffer from inherent hardware limitations, which restrict their ability to implement comprehensive security measures and increase their exposure to vulnerability attacks. This paper proposes a novel Network Intrusion Prevention System that utilises a SelfOrganizing Incremental Neural Network along with a Support Vector Machine. Due to its structure, the proposed system provides a security solution that does not rely on signatures …
We present a probabilistic model of an intrusion in a renewal process. Given a process and a sequence of events, an intrusion is a subsequence of events that is not produced by the process. Applications of the model are, for example, online payment fraud with the fraudster taking over a user's account and performing payments on the user's behalf, or unexpected equipment failures due to unintended use. We adopt Bayesian approach to infer the probability of an intrusion in a sequence of events, a MAP subsequence of events constituting the intrusion, and the marginal probability of each event in …
The network security analyzers use intrusion detection systems (IDSes) to distinguish malicious traffic from benign ones. The deep learning-based IDSes are proposed to auto-extract high-level features and eliminate the time-consuming and costly signature extraction process. However, this new generation of IDSes still suffers from a number of challenges. One of the main issues of an IDS is facing traffic concept drift which manifests itself as new (i.e., zero-day) attacks, in addition to the changing behavior of benign users/applications. Furthermore, a practical DL-based IDS needs to be confor…
Neural networks have become an increasingly popular solution for network intrusion detection systems (NIDS). Their capability of learning complex patterns and behaviors make them a suitable solution for differentiating between normal traffic and network attacks. However, a drawback of neural networks is the amount of resources needed to train them. Many network gateways and routers devices, which could potentially host an NIDS, simply do not have the memory or processing power to train and sometimes even execute such models. More importantly, the existing neural network solutions are trained i…
Few-Shot Network Intrusion Detection Using Online Triplet Mining
Department of Electronics and Electrical Engineering, University of Strathclyde（斯特拉斯克莱德大学电子与电气工程系）；Faculty of Computer and Information Sciences, Ain Shams University（爱思曼大学计算机与信息科学学院）；Faculty of Electrical Engineering, Czech Technical University（捷克技术大学电气工程学院）
As one of the solutions to intrusion detection problems, Artificial Immune Systems (AIS) have shown their advantages. Unlike genetic algorithms, there is no one archetypal AIS, instead there are four major paradigms. Among them, the Dendritic Cell Algorithm (DCA) has produced promising results in various applications. The aim of this chapter is to demonstrate the potential for the DCA as a suitable candidate for intrusion detection problems. We review some of the commonly used AIS paradigms for intrusion detection problems and demonstrate the advantages of one particular algorithm, the DCA. In…
Prevention of cyber attacks on the critical network resources has become an important issue as the traditional Intrusion Detection Systems (IDSs) are no longer effective due to the high volume of network traffic and the deceptive patterns of network usage employed by the attackers. Lack of sufficient amount of labeled observations for the training of IDSs makes the semi-supervised IDSs a preferred choice. We propose a semi-supervised IDS by extending a data analysis technique known as Logical Analysis of Data, or LAD in short, which was proposed as a supervised learning approach. LAD uses part…
Security of computers and the networks that connect them is increasingly becoming of great significance. Computer security is defined as the protection of computing systems against threats to confidentiality, integrity, and availability. There are two types of intruders: the external intruders who are unauthorized users of the machines they attack, and internal intruders, who have permission to access the system with some restrictions. Due to the fact that it is more and more improbable to a system administrator to recognize and manually intervene to stop an attack, there is an increasing reco…
Over the recent years, IP and email spoofing gained much importance for security concerns due to the current changes in manipulating the system performance in different online environments. Intrusion Detection System (IDS) has been used to secure these environments for sharing their data over network and host based IDS approaches. However, the rapid growth of intrusion events over Internet and local area network become responsible for the distribution of different threats and vulnerabilities in the computing systems. The current signature detection approach used by IDS, detects unclear actions…
Smartphone technology is more and more becoming the predominant communication tool for people across the world. People use their smartphones to keep their contact data, to browse the internet, to exchange messages, to keep notes, carry their personal files and documents, etc. Users while browsing are also capable of shopping online, thus provoking a need to type their credit card numbers and security codes. As the smartphones are becoming widespread so do the security threats and vulnerabilities facing this technology. Recent news and articles indicate huge increase in malware and viruses for …
Multi-Level Distributional Entropy from Flow Summary Statistics for Explainable Network Intrusion Detection
Centre for Intelligent Cloud Computing, CoE for Advanced Cloud, Faculty of Information Science and Technology, Multimedia University（智能云计算中心、先进云技术中心、信息科学与技术学院、多媒体大学）；Department of Communication Technology and Networking, Faculty of Computer Science and Information Technology, Universiti Putra Malaysia（通信技术与网络系、计算机科学与信息技术学院、马来西亚国立大学）；Laboratory of Computational Science and Mathematical Physics, Institute for Mathematical Research, Universiti Putra Malaysia（计算科学与数学物理实验室、数学研究所、马来西亚国立大学）
