Malware developers use combinations of techniques such as compression, encryption, and obfuscation to bypass anti-virus software. Malware with anti-analysis technologies can bypass AI-based anti-virus software and malware analysis tools. Therefore, classifying pack files is one of the big challenges. Problems arise if the malware classifiers learn packers' features, not those of malware. Training the models with unintended erroneous data turn into poisoning attacks, adversarial attacks, and evasion attacks. Therefore, researchers should consider packing to build appropriate malware classifier models. In this paper, we propose a multi-step framework for classifying and identifying packed samples which consists of pseudo-optimal feature selection, machine learning-based classifiers, and packer identification steps. In the first step, we use the CART algorithm and the permutation importance to preselect important 20 features. In the second step, each model learns 20 preselected features for classifying the packed files with the highest performance. As a result, the XGBoost, which learned the features preselected by XGBoost with the permutation importance, showed the highest performance of any other experiment scenarios with an accuracy of 99.67%, an F1-Score of 99.46%, and an area under the curve (AUC) of 99.98%. In the third step, we propose a new approach that can identify packers only for samples classified as Well-Known Packed.
A Multi-task Mixture of Experts Framework for Malware Classification, Packing Detection, and Family Attribution
Department of Computer Applications, Cochin University of Science and Technology, India（计算机应用系，科钦大学科学与技术学院，印度）；Department of Electrical, Computer and Biomedical Engineering, University of Pavia, Italy（电气、计算机和生物医学工程系，帕维亚大学，意大利）
Current malware detection and classification approaches generally rely on time consuming and knowledge intensive processes to extract patterns (signatures) and behaviors from malware, which are then used for identification. Moreover, these signatures are often limited to local, contiguous sequences within the data whilst ignoring their context in relation to each other and throughout the malware file as a whole. We present a Deep Learning based malware classification approach that requires no expert domain knowledge and is based on a purely data driven approach for complex pattern and feature …
Guarding Organizations Against Malware Risk: A Novel Graph-Based Malware Detection Method
School of Management, Fudan University（复旦大学管理学院）；School of Information Management and Engineering, Shanghai University of Finance and Economics（上海财经大学信息管理与工程学院）；Lerner College of Business and Economics, University of Delaware（特拉华大学勒纳工商经济学院）
Leopard Mobile Inc.（猎豹移动公司）；National Cheng Kung University（国立成功大学）
The influence of Deep Learning on image identification and natural language processing has attracted enormous attention globally. The convolution neural network that can learn without prior extraction of features fits well in response to the rapid iteration of Android malware. The traditional solution for detecting Android malware requires continuous learning through pre-extracted features to maintain high performance of identifying the malware. In order to reduce the manpower of feature engineering prior to the condition of not to extract pre-selected features, we have developed a coloR-inspi…
MAD-OOD: A Deep Learning Cluster-Driven Framework for an Out-of-Distribution Malware Detection and Classification
Dept. of Computer Science The University of Texas at El Paso Texas, USA（计算机科学系 乌德勒支理工大学 美国德克萨斯州）；Dept. of Computer Science University of North Carolina North Carolina, USA（计算机科学系 北卡罗来纳大学 美国北卡罗来纳州）
Malware is a pervasive problem in both personal computing devices and distributed computing systems. Identification of malware variants and their families others a great benefit in early detection resulting in a reduction of the analyses time needed. In order to classify malware, most of the current approaches are based on the analysis of the unpacked and unencrypted binaries. However, most of the unpacking solutions in the literature have a low unpacking rate. This results in a low contribution towards the identification of transferred code and re-used code. To develop a new malware analysis …
Al-Azhar University（爱资哈尔大学）；Egyptian Chinese University（埃中大学）
Android malware is one of the most dangerous threats on the internet, and it's been on the rise for several years. Despite significant efforts in detecting and classifying android malware from innocuous android applications, there is still a long way to go. As a result, there is a need to provide a basic understanding of the behavior displayed by the most common Android malware categories and families. Each Android malware family and category has a distinct objective. As a result, it has impacted every corporate area, including healthcare, banking, transportation, government, and e-commerce. I…
Plymouth University（普利茅斯大学）；Open University of Cyprus（塞浦路斯开放大学）
Digital investigators often get involved with cases, which seemingly point the responsibility to the person to which the computer belongs, but after a thorough examination malware is proven to be the cause, causing loss of precious time. Whilst Anti-Virus (AV) software can assist the investigator in identifying the presence of malware, with the increase in zero-day attacks and errors that exist in AV tools, this is something that cannot be relied upon. The aim of this paper is to investigate the behaviour of malware upon various Windows operating system versions in order to determine and corre…
Identifying the tasks a given piece of malware was designed to perform (e.g. logging keystrokes, recording video, establishing remote access, etc.) is a difficult and time-consuming operation that is largely human-driven in practice. In this paper, we present an automated method to identify malware tasks. Using two different malware collections, we explore various circumstances for each - including cases where the training data differs significantly from test; where the malware being evaluated employs packing to thwart analytical techniques; and conditions with sparse training data. We find th…
Computers are widely used today by most people. Internet based applications, like ecommerce or ebanking attracts criminals, who using sophisticated techniques, tries to introduce malware on the victim computer. But not only computer users are in risk, also smartphones or smartwatch users, smart cities, Internet of Things devices, etc. Different techniques has been tested against malware. Currently, pattern matching is the default approach in antivirus software. Also, Machine Learning is successfully being used. Continuing this trend, in this article we propose an anomaly based method using the…
Packed Malware Detection Using Grayscale Binary-to-Image Representations
Canadian Institute for Cybersecurity (CIC)（加拿大网络安全研究所）；University of New Brunswick（新不伦瑞克大学）；Behaviour-Centric Cybersecurity Center (BCCC)（以行为为中心的网络安全中心）；York University（约克大学）
Mayachitra, Inc.（玛雅奇特拉公司）；U.S. Naval Research Laboratory（美国海军研究实验室）；UC Santa Barbara（加州大学圣巴巴拉分校）
The number of malware is constantly on the rise. Though most new malware are modifications of existing ones, their sheer number is quite overwhelming. In this paper, we present a novel system to visualize and map millions of malware to points in a 2-dimensional (2D) spatial grid. This enables visualizing relationships within large malware datasets that can be used to develop triage solutions to screen different malware rapidly and provide situational awareness. Our approach links two visualizations within an interactive display. Our first view is a spatial point-based visualization of similari…
Malware and other suspicious software often hide behaviors and components behind logic bombs and context-sensitive execution paths. Uncovering these is essential to react against modern threats, but current solutions are not ready to detect these paths in a completely automated manner. To bridge this gap, we propose the Malware Multiverse (MalVerse), a solution able to inspect multiple execution paths via symbolic execution aiming to discover function inputs and returns that trigger malicious behaviors. MalVerse automatically patches the context-sensitive functions with the identified symbolic…
Malware detection have used machine learning to detect malware in programs. These applications take in raw or processed binary data to neural network models to classify as benign or malicious files. Even though this approach has proven effective against dynamic changes, such as encrypting, obfuscating and packing techniques, it is vulnerable to specific evasion attacks where that small changes in the input data cause misclassification at test time. This paper proposes a new approach: MDEA, an Adversarial Malware Detection model uses evolutionary optimization to create attack samples to make th…
Be it for a malicious or legitimate purpose, packing, a transformation that consists in applying various operations like compression or encryption to a binary file, i.e. for making reverse engineering harder or obfuscating code, is widely employed since decades already. Particularly in the field of malware analysis where a stumbling block is evasion, it has proven effective and still gives a hard time to scientists who want to efficiently detect it. While already extensively covered in the scientific literature, it remains an open issue especially when considering its detection time and accura…
Since Google unveiled Android OS for smartphones, malware are thriving with 3Vs, i.e. volume, velocity, and variety. A recent report indicates that one out of every five business/industry mobile application leaks sensitive personal data. Traditional signature/heuristic-based malware detection systems are unable to cope up with current malware challenges and thus threaten the Android ecosystem. Therefore recently researchers have started exploring machine learning and deep learning based malware detection systems. In this paper, we performed a comprehensive feature analysis to identify the sign…
Research Center of Cyber Intelligence and Information Security (CIS)（网络情报与信息安全研究中心（CIS））；Sapienza University of Rome（罗马第一大学）
In the last decade, a new class of cyber-threats has emerged. This new cybersecurity adversary is known with the name of "Advanced Persistent Threat" (APT) and is referred to different organizations that in the last years have been "in the center of the eye" due to multiple dangerous and effective attacks targeting financial and politic, news headlines, embassies, critical infrastructures, TV programs, etc. In order to early identify APT related malware, a semi-automatic approach for malware samples analysis is needed. In our previous work we introduced a "malware triage" step for a semi-autom…
In response to the volume and sophistication of malicious software or malware, security investigators rely on dynamic analysis for malware detection to thwart obfuscation and packing issues. Dynamic analysis is the process of executing binary samples to produce reports that summarise their runtime behaviors. The investigator uses these reports to detect malware and attribute threat type leveraging manually chosen features. However, the diversity of malware and the execution environments makes manual approaches not scalable because the investigator needs to manually engineer fingerprinting feat…
Packing is an obfuscation technique widely used by malware to hide the content and behavior of a program. Much prior research has explored how to detect whether a program is packed. This research includes a broad variety of approaches such as entropy analysis, syntactic signatures and more recently machine learning classifiers using various features. However, no robust results have indicated which algorithms perform best, or which features are most significant. This is complicated by considering how to evaluate the results since accuracy, cost, generalization capabilities, and other measures a…
Semi-supervised Classification of Malware Families Under Extreme Class Imbalance via Hierarchical Non-Negative Matrix Factorization with Automatic Model Selection
本文提出一种名为 HNMFk Classifier 的分层半监督算法，基于带自动模型选择的非负矩阵分解，通过支持弃权（不执行）预测，在极端类别不平衡和标注数据稀缺的情况下有效分类恶意软件家族并识别新型家族，在 EMBER-2018 数据集上取得 0.80 的 F1 分数。
The Android mining sandbox approach consists in running dynamic analysis tools on a benign version of an Android app and recording every call to sensitive APIs. Later, one can use this information to (a) prevent calls to other sensitive APIs (those not previously recorded) or (b) run the dynamic analysis tools again in a different version of the app -- in order to identify possible malicious behavior. Although the use of dynamic analysis for mining Android sandboxes has been empirically investigated before, little is known about the potential benefits of combining static analysis with the mini…
Privacy-preserving Preselection for Face Identification Based on Packing
College of Electronics and Information Engineering（电子信息工程学院）；the Provincial Key Laboratory of Intelligent Communication and Digital Society Governance（智能通信与数字社会治理省级重点实验室）；Shenzhen University（深圳大学）；Sanya Institute of Hunan University of Science and Technology（湖南科技大学三亚学院）；School of Computer Science and Cyber Engineering（计算机科学与网络工程学院）；Guangzhou University（广州大学）；School of Computer Science and Technology（计算机科学与技术学院）；Changsha University of Science and Technology（长沙理工大学）；School of Civil and Environmental Engineering（土木与环境工程学院）
Bredesen Center（布雷德森中心）；Oak Ridge National Laboratory（橡树岭国家实验室）；Tennessee Tech University（田纳西理工大学）；Pacific Northwest National Laboratory（太平洋西北国家实验室）
Standardized file formats play a key role in the development and use of computer software. However, it is possible to abuse standardized file formats by creating a file that is valid in multiple file formats. The resulting polyglot (many languages) file can confound file format identification, allowing elements of the file to evade analysis.This is especially problematic for malware detection systems that rely on file format identification for feature extraction. File format identification processes that depend on file signatures can be easily evaded thanks to flexibility in the format specifi…
University of Murcia（穆尔西亚大学）；armasuisse Science & Technology（瑞士国防采购与技术局科学与技术部）；University of Zurich（苏黎世大学）
In the last years, the number of IoT devices deployed has suffered an undoubted explosion, reaching the scale of billions. However, some new cybersecurity issues have appeared together with this development. Some of these issues are the deployment of unauthorized devices, malicious code modification, malware deployment, or vulnerability exploitation. This fact has motivated the requirement for new device identification mechanisms based on behavior monitoring. Besides, these solutions have recently leveraged Machine and Deep Learning techniques due to the advances in this field and the increase…
Through the generalization of deep learning, the research community has addressed critical challenges in the network security domain, like malware identification and anomaly detection. However, they have yet to discuss deploying them on Internet of Things (IoT) devices for day-to-day operations. IoT devices are often limited in memory and processing power, rendering the compute-intensive deep learning environment unusable. This research proposes a way to overcome this barrier by bypassing feature engineering in the deep learning pipeline and using raw packet data as input. We introduce a featu…
University of Washington（华盛顿大学）；Georgia Institute of Technology（佐治亚理工学院）
Binary analysis of software is a critical step in cyber forensics applications such as program vulnerability assessment and malware detection. This involves interpreting instructions executed by software and often necessitates converting the software's binary file data to assembly language. The conversion process requires information about the binary file's target instruction set architecture (ISA). However, ISA information might not be included in binary files due to compilation errors, partial downloads, or adversarial corruption of file metadata. Machine learning (ML) is a promising methodo…
The generalization of deep learning has helped us, in the past, address challenges such as malware identification and anomaly detection in the network security domain. However, as effective as it is, scarcity of memory and processing power makes it difficult to perform these tasks in Internet of Things (IoT) devices. This research finds an easy way out of this bottleneck by depreciating the need for feature engineering and subsequent processing in machine learning techniques. In this study, we introduce a Featureless machine learning process to perform anomaly detection. It uses unprocessed by…
Numerous malware families rely on domain generation algorithms (DGAs) to establish a connection to their command and control (C2) server. Counteracting DGAs, several machine learning classifiers have been proposed enabling the identification of the DGA that generated a specific domain name and thus triggering targeted remediation measures. However, the proposed state-of-the-art classifiers are based on deep learning models. The black box nature of these makes it difficult to evaluate their reasoning. The resulting lack of confidence makes the utilization of such models impracticable. In this p…
