Robust network security systems are essential to prevent and mitigate the harming effects of the ever-growing occurrence of network attacks. In recent years, machine learning-based systems have gain popularity for network security applications, usually considering the application of shallow models, which rely on the careful engineering of expert, handcrafted input features. The main limitation of this approach is that handcrafted features can fail to perform well under different scenarios and types of attacks. Deep Learning (DL) models can solve this limitation using their ability to learn feature representations from raw, non-processed data. In this paper we explore the power of DL models on the specific problem of detection and classification of malware network traffic. As a major advantage with respect to the state of the art, we consider raw measurements coming directly from the stream of monitored bytes as input to the proposed models, and evaluate different raw-traffic feature representations, including packet and flow-level ones. We introduce DeepMAL, a DL model which is able to capture the underlying statistics of malicious traffic, without any sort of expert handcrafted features. Using publicly available traffic traces containing different families of malware traffic, we show that DeepMAL can detect and classify malware flows with high accuracy, outperforming traditional, shallow-like models.
Northeastern University（东北大学）；Michigan Technological University（密歇根理工大学）；University of Florida（佛罗里达大学）；Federal University of Parana（巴拉那联邦大学）；Cognization Lab（认知实验室）
Existing malware detectors on safety-critical devices have difficulties in runtime detection due to the performance overhead. In this paper, we introduce PROPEDEUTICA, a framework for efficient and effective real-time malware detection, leveraging the best of conventional machine learning (ML) and deep learning (DL) techniques. In PROPEDEUTICA, all software start execution are considered as benign and monitored by a conventional ML classifier for fast detection. If the software receives a borderline classification from the ML detector (e.g. the software is 50% likely to be benign and 50% likel…
With the increase of IoT devices and technologies coming into service, Malware has risen as a challenging threat with increased infection rates and levels of sophistication. Without strong security mechanisms, a huge amount of sensitive data is exposed to vulnerabilities, and therefore, easily abused by cybercriminals to perform several illegal activities. Thus, advanced network security mechanisms that are able of performing a real-time traffic analysis and mitigation of malicious traffic are required. To address this challenge, we are proposing a novel IoT malware traffic analysis approach u…
University of Oxford（牛津大学）；Sapienza University of Rome（罗马第一大学）
Economic incentives encourage malware authors to constantly develop new, increasingly complex malware to steal sensitive data or blackmail individuals and companies into paying large ransoms. In 2017, the worldwide economic impact of cyberattacks is estimated to be between 445 and 600 billion USD, or 0.8% of global GDP. Traditionally, one of the approaches used to defend against malware is network traffic analysis, which relies on network data to detect the presence of potentially malicious software. However, to keep up with increasing network speeds and amount of traffic, network analysis is …
Internet traffic classification is widely used to facilitate network management. It plays a crucial role in Quality of Services (QoS), Quality of Experience (QoE), network visibility, intrusion detection, and traffic trend analyses. While there is no theoretical guarantee that deep learning (DL)-based solutions perform better than classic machine learning (ML)-based ones, DL-based models have become the common default. This paper compares well-known DL-based and ML-based models and shows that in the case of malicious traffic classification, state-of-the-art DL-based solutions do not necessaril…
With malware detection techniques increasingly adopting machine learning approaches, the creation of precise training sets becomes more and more important. Large data sets of realistic web traffic, correctly classified as benign or malicious are needed, not only to train classic and deep learning algorithms, but also to serve as evaluation benchmarks for existing malware detection products. Interestingly, despite the vast number and versatility of threats a user may encounter when browsing the web, actual malicious content is often hard to come by, since prerequisites such as browser and opera…
Identifying threats in a network traffic flow which is encrypted is uniquely challenging. On one hand it is extremely difficult to simply decrypt the traffic due to modern encryption algorithms. On the other hand, passing such an encrypted stream through pattern matching algorithms is useless because encryption ensures there aren't any. Moreover, evaluating such models is also difficult due to lack of labeled benign and malware datasets. Other approaches have tried to tackle this problem by employing observable meta-data gathered from the flow. We try to augment this approach by extending it t…
Internet of Things devices have seen a rapid growth and popularity in recent years with many more ordinary devices gaining network capability and becoming part of the ever growing IoT network. With this exponential growth and the limitation of resources, it is becoming increasingly harder to protect against security threats such as malware due to its evolving faster than the defence mechanisms can handle with. The traditional security systems are not able to detect unknown malware as they use signature-based methods. In this paper, we aim to address this issue by introducing a novel IoT malwar…
Malware is constantly evolving. Although existing countermeasures have success in malware detection, corresponding counter-countermeasures are always emerging. In this study, a counter-countermeasure that avoids network-based detection approaches by camouflaging malicious traffic as an innocuous protocol is presented. The approach includes two steps: Traffic format transformation and side-channel massage (SCM). Format transforming encryption (FTE) translates protocol syntax to mimic another innocuous protocol while SCM obscures traffic side-channels. The proposed approach is illustrated by tra…
Traditional techniques to detect malware infections were not meant to be used by the end-user and current malware removal tools and security software cannot handle the heterogeneity of IoT devices. In this paper, we design, develop and evaluate a tool, called NURSE, to fill this information gap, i.e., enabling end-users to detect IoT-malware infections in their home networks. NURSE follows a modular approach to analyze IoT traffic as captured by means of an ARP spoofing technique which does not require any network modification or specific hardware. Thus, NURSE provides zero-configuration IoT t…
Nowadays, Android is the most dominant operating system in the mobile ecosystem, with billions of people using its apps daily. As expected, this trend did not go unnoticed by miscreants, and Android became the favorite platform for discovering new victims through malicious apps. Moreover, these apps have become so sophisticated that they can bypass anti-malware measures to protect the users. Therefore, it is safe to admit that traditional anti-malware techniques have become cumbersome, sparking the urge to develop an efficient way to detect Android malware. This paper presents hybrid-Flacon, a…
The various types of communication technologies and mobility features in Internet of Things (IoT) on the one hand enable fruitful and attractive applications, but on the other hand facilitates malware propagation, thereby raising new challenges on handling IoT-empowered malware for cyber security. Comparing with the malware propagation control scheme in traditional wireless networks where nodes can be directly repaired and secured, in IoT, compromised end devices are difficult to be patched. Alternatively, blocking malware via patching intermediate nodes turns out to be a more feasible and pra…
Research and development of techniques which detect or remediate malicious network activity require access to diverse, realistic, contemporary data sets containing labeled malicious connections. In the absence of such data, said techniques cannot be meaningfully trained, tested, and evaluated. Synthetically produced data containing fabricated or merged network traffic is of limited value as it is easily distinguishable from real traffic by even simple machine learning (ML) algorithms. Real network data is preferable, but while ubiquitous is broadly both sensitive and lacking in ground truth la…
The widespread adoption of Internet of Things has led to many security issues. Post the Mirai-based DDoS attack in 2016 which compromised IoT devices, a host of new malware using Mirai's leaked source code and targeting IoT devices have cropped up, e.g. Satori, Reaper, Amnesia, Masuta etc. These malware exploit software vulnerabilities to infect IoT devices instead of open TELNET ports (like Mirai) making them more difficult to block using existing solutions such as firewalls. In this research, we present EDIMA, a distributed modular solution which can be used towards the detection of IoT malw…
State of the art deep learning techniques are known to be vulnerable to evasion attacks where an adversarial sample is generated from a malign sample and misclassified as benign. Detection of encrypted malware command and control traffic based on TCP/IP flow features can be framed as a learning task and is thus vulnerable to evasion attacks. However, unlike e.g. in image processing where generated adversarial samples can be directly mapped to images, going from flow features to actual TCP/IP packets requires crafting the sequence of packets, with no established approach for such crafting and a…
Malicious software (malware) poses an increasing threat to the security of communication systems as the number of interconnected mobile devices increases exponentially. While some existing malware detection and classification approaches successfully leverage network traffic data, they treat network flows between pairs of endpoints independently and thus fail to leverage rich communication patterns present in the complete network. Our approach first extracts flow graphs and subsequently classifies them using a novel edge feature-based graph neural network model. We present three variants of our…
Security has become ubiquitous in every domain today as newly emerging malware pose an ever-increasing perilous threat to systems. Consequently, honeypots are fast emerging as an indispensible forensic tool for the analysis of malicious network traffic. Honeypots can be considered to be traps for hackers and intruders and are generally deployed complimentary to Intrusion Detection Systems (IDS) and Intrusion Prevention Systems (IPS) in a network. They help system administrators perform a rigorous analysis of external and internal attacks on their networks. They are also used by security firms …
University of Potsdam（波茨坦大学）；Cisco R&D（思科研发部）；Czech Technical University in Prague（布拉格捷克理工大学）
Detection of malware-infected computers and detection of malicious web domains based on their encrypted HTTPS traffic are challenging problems, because only addresses, timestamps, and data volumes are observable. The detection problems are coupled, because infected clients tend to interact with malicious domains. Traffic data can be collected at a large scale, and antivirus tools can be used to identify infected clients in retrospect. Domains, by contrast, have to be labeled individually after forensic analysis. We explore transfer learning based on sluice networks; this allows the detection m…
Center for Computational Engineering and Networking (CEN)（计算工程与网络中心（CEN））；Amrita School of Engineering（阿姆里塔工程学院）；Amrita Vishwa Vidyapeetham（阿姆里塔大学）
Deep learning is an advanced model of traditional machine learning. This has the capability to extract optimal feature representation from raw input samples. This has been applied towards various use cases in cyber security such as intrusion detection, malware classification, android malware detection, spam and phishing detection and binary analysis. This paper outlines the survey of all the works related to deep learning based solutions for various cyber security use cases. Keywords: Deep learning, intrusion detection, malware detection, Android malware detection, spam & phishing detection, t…
Network intrusion detection systems and antivirus software are essential in detecting malicious network traffic and attacks such as denial-of-service and malwares. Each attack, worm or virus has its own distinctive signature. Signature-based intrusion detection and antivirus systems depend on pattern matching to look for possible attack signatures. Pattern matching is a very complex task, which requires a lot of time, memory and computing resources. Software-based intrusion detection is not fast enough to match high network speeds and the increasing number of attacks. In this paper, we propose…
Recent self-propagating malware (SPM) campaigns compromised hundred of thousands of victim machines on the Internet. It is challenging to detect these attacks in their early stages, as adversaries utilize common network services, use novel techniques, and can evade existing detection mechanisms. We propose PORTFILER (PORT-Level Network Traffic ProFILER), a new machine learning system applied to network traffic for detecting SPM attacks. PORTFILER extracts port-level features from the Zeek connection logs collected at a border of a monitored network, applies anomaly detection techniques to iden…
Singapore University of Technology and Design（新加坡科技设计大学）；National University of Singapore（新加坡国立大学）；University of Sydney（悉尼大学）
In this work, we present a lightweight IoT botnet detection solution, EDIMA, which is designed to be deployed at the edge gateway installed in home networks and targets early detection of botnets prior to the launch of an attack. EDIMA includes a novel two-stage Machine Learning (ML)-based detector developed specifically for IoT bot detection at the edge gateway. The ML-based bot detector first employs ML algorithms for aggregate traffic classification and subsequently Autocorrelation Function (ACF)-based tests to detect individual bots. The EDIMA architecture also comprises a malware traffic …
Shiraz University of Technology（设拉子理工大学）；University of Surrey（萨里大学）
In recent years, malware detection has become an active research topic in the area of Internet of Things (IoT) security. The principle is to exploit knowledge from large quantities of continuously generated malware. Existing algorithms practice available malware features for IoT devices and lack real-time prediction behaviors. More research is thus required on malware detection to cope with real-time misclassification of the input IoT data. Motivated by this, in this paper we propose an adversarial self-supervised architecture for detecting malware in IoT networks, SETTI, considering samples o…
Packet Inspection Transformer: A Self-Supervised Journey to Unseen Malware Detection with Few Samples
Department of Intelligent Systems and Robotics, University of West Florida, Pensacola, FL, USA（智能系统与机器人系，西佛罗里达大学，佛罗里达州彭萨科大，USA）；Florida Institute For Human and Machine Cognition (IHMC), Pensacola, FL, USA（佛罗里达人类与机器认知研究所（IHMC），佛罗里达州彭萨科大，USA）；Center for Cybersecurity, University of West Florida, Pensacola, FL, USA（网络安全中心，西佛罗里达大学，佛罗里达州彭萨科大，USA）
University of Massachusetts Lowell（马萨诸塞大学洛厄尔分校）；Intel Corporation（英特尔公司）；Elizabethtown College（伊丽莎白敦学院）
Classifying network traffic is the basis for important network applications. Prior research in this area has faced challenges on the availability of representative datasets, and many of the results cannot be readily reproduced. Such a problem is exacerbated by emerging data-driven machine learning based approaches. To address this issue, we provide three open datasets containing almost 1.3M labeled flows in total, with flow features and anonymized raw packets, for the research community. We focus on broad aspects in network traffic analysis, including both malware detection and application cla…
The widespread adoption of Internet of Things has led to many security issues. Recently, there have been malware attacks on IoT devices, the most prominent one being that of Mirai. IoT devices such as IP cameras, DVRs and routers were compromised by the Mirai malware and later large-scale DDoS attacks were propagated using those infected devices (bots) in October 2016. In this research, we develop a network-based algorithm which can be used to detect IoT bots infected by Mirai or similar malware in large-scale networks (e.g. ISP network). The algorithm particularly targets bots scanning the ne…
École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University of Murcia（穆尔西亚大学）；University of Zurich UZH（苏黎世大学）；Cyber-Defence Campus, armasuisse Science & Technology（瑞士国防部科技局网络防御校区）
This work investigates the possibilities enabled by federated learning concerning IoT malware detection and studies security issues inherent to this new learning paradigm. In this context, a framework that uses federated learning to detect malware affecting IoT devices is presented. N-BaIoT, a dataset modeling network traffic of several real IoT devices while affected by malware, has been used to evaluate the proposed framework. Both supervised and unsupervised federated models (multi-layer perceptron and autoencoder) able to detect malware affecting seen and unseen IoT devices of N-BaIoT have…
Network traffic analytics technology is a cornerstone for cyber security systems. We demonstrate its use through three popular and contemporary cyber security applications in intrusion detection, malware analysis and botnet detection. However, automated traffic analytics faces the challenges raised by big traffic data. In terms of big data's three characteristics --- volume, variety and velocity, we review three state of the art techniques to mitigate the key challenges including real-time traffic classification, unknown traffic classification, and efficiency of classifiers. The new techniques…
A Transformer-Based Framework for Payload Malware Detection and Classification
University of West Florida（西佛罗里达大学）；Florida Institute For Human and Machine Cognition (IHMC)（佛罗里达人机认知研究所）；Center for Cybersecurity, University of West Florida（西佛罗里达大学网络安全中心）
Malware abuses TLS to encrypt its malicious traffic, preventing examination by content signatures and deep packet inspection. Network detection of malicious TLS flows is an important, but challenging, problem. Prior works have proposed supervised machine learning detectors using TLS features. However, by trying to represent all malicious traffic, supervised binary detectors produce models that are too loose, thus introducing errors. Furthermore, they do not distinguish flows generated by different malware. On the other hand, supervised multi-class detectors produce tighter models and can class…
The upcoming Internet of things (IoT) is foreseen to encompass massive numbers of connected devices, smart objects, and cyber-physical systems. Due to the large-scale and massive deployment of devices, it is deemed infeasible to safeguard 100% of the devices with state-of-the-art security countermeasures. Hence, large-scale IoT has inevitable loopholes for network intrusion and malware infiltration. Even worse, exploiting the high density of devices and direct wireless connectivity, malware infection can stealthily propagate through susceptible (i.e., unsecured) devices and form an epidemic ou…
Many techniques have been proposed for quickly detecting and containing malware-generated network attacks such as large-scale denial of service attacks; unfortunately, much damage is already done within the first few minutes of an attack, before it is identified and contained. There is a need for an early warning system that can predict attacks before they actually manifest, so that upcoming attacks can be prevented altogether by blocking the hosts that are likely to engage in attacks. However, blocking responses may disrupt legitimate processes on blocked hosts; in order to minimise user inco…
Cybersecurity is essential, and attacks are rapidly growing and getting more challenging to detect. The traditional Firewall and Intrusion Detection system, even though it is widely used and recommended but it fails to detect new attacks, zero-day attacks, and traffic patterns that do not match with any configured rules. Therefore, Machine Learning (ML) can be an efficient and cost-reduced solution in cybersecurity. We used Netflow datasets to extract features after applying data analysis. Then, a selection process has been applied to compare these features with one another. Our experiments fo…
As the number of heterogenous IP-connected devices and traffic volume increase, so does the potential for security breaches. The undetected exploitation of these breaches can bring severe cybersecurity and privacy risks. Anomaly-based \acp{IDS} play an essential role in network security. In this paper, we present a practical unsupervised anomaly-based deep learning detection system called ARCADE (Adversarially Regularized Convolutional Autoencoder for unsupervised network anomaly DEtection). With a convolutional \ac{AE}, ARCADE automatically builds a profile of the normal traffic using a subse…
Compared to cryptography, steganography is a less discussed domain. However, there is a recent trend of exploiting various information hiding techniques to empower malware, for instance to bypass security frameworks of mobile devices or to exfiltrate sensitive data. This is mostly due to the need to counteract increasingly sophisticated security mechanisms, such as code analysis, runtime countermeasures, or real-time traffic inspection tools. In this perspective, this paper presents malware exploiting information hiding in a broad sense, i.e., it does not focus on classical covert channels, bu…
International Hellenic University（国际希腊大学）；Democritus University of Thrace（德谟克利特大学）；Hellenic National Defence General Staff（希腊国防参谋本部）；University of Aegean（爱琴大学）
Attackers are perpetually modifying their tactics to avoid detection and frequently leverage legitimate credentials with trusted tools already deployed in a network environment, making it difficult for organizations to proactively identify critical security risks. Network traffic analysis products have emerged in response to attackers relentless innovation, offering organizations a realistic path forward for combatting creative attackers. Additionally, thanks to the widespread adoption of cloud computing, Device Operators processes, and the Internet of Things, maintaining effective network vis…
The Linux shell is a command-line interpreter that provides users with a command interface to the operating system, allowing them to perform a variety of functions. Although very useful in building capabilities at the edge, the Linux shell can be exploited, giving adversaries a prime opportunity to use them for malicious activities. With access to IoT devices, malware authors can abuse the Linux shell of those devices to propagate infections and launch large-scale attacks, e.g., DDoS. In this work, we provide a first look at shell commands used in Linux-based IoT malware towards detection. We …
Enterprise Networks are growing in scale and complexity, with heterogeneous connected assets needing to be secured in different ways. Nevertheless, virtually all connected assets use the Domain Name System (DNS) for address resolution, and DNS has thus become a convenient vehicle for attackers to covertly perform Command and Control (C&C) communication, data theft, and service disruption across a wide range of assets. Enterprise security appliances that monitor network traffic typically allow all DNS traffic through as it is vital for accessing any web service; they may at best match against a…
Imperial College London（帝国理工学院）；Kensho Technologies（Kensho科技公司）；Tencent（腾讯）
Botnets and malware continue to avoid detection by static rules engines when using domain generation algorithms (DGAs) for callouts to unique, dynamically generated web addresses. Common DGA detection techniques fail to reliably detect DGA variants that combine random dictionary words to create domain names that closely mirror legitimate domains. To combat this, we created a novel hybrid neural network, Bilbo the `bagging` model, that analyses domains and scores the likelihood they are generated by such algorithms and therefore are potentially malicious. Bilbo is the first parallel usage of a …
Cybercrime is increasing at a faster pace and sometimes causes billions of dollars of business- losses so investigating attackers after commitment is of utmost importance and become one of the main concerns of network managers. Network forensics as the process of Collecting, identifying, extracting and analyzing data and systematically monitoring traffic of network is one of the main requirements in detection and tracking of criminals. In this paper, we propose an architecture for network forensic system. Our proposed architecture consists of five main components: collection and indexing, data…
