Mind the Gap: On Bridging the Semantic Gap between Machine Learning and Information Security
Sandia National Laboratories（桑迪亚国家实验室）；New Mexico Tech（新墨西哥理工学院）
Michael R. Smith, Nicholas T. Johnson, Joe B. Ingram, Armida J. Carbajal, Ramyaa Ramyaa, Evelyn Domschot, Christopher C. Lamb, Stephen J. Verzi, W. Philip Kegelmeyer
Despite the potential of Machine learning (ML) to learn the behavior of malware, detect novel malware samples, and significantly improve information security (InfoSec) we see few, if any, high-impact ML techniques in deployed systems, notwithstanding multiple reported successes in open literature. We hypothesize that the failure of ML in making high-impacts in InfoSec are rooted in a disconnect between the two communities as evidenced by a semantic gap---a difference in how executables are described (e.g. the data and features extracted from the data). Specifically, current datasets and representations used by ML are not suitable for learning the behaviors of an executable and differ significantly from those used by the InfoSec community. In this paper, we survey existing datasets used for classifying malware by ML algorithms and the features that are extracted from the data. We observe that: 1) the current set of extracted features are primarily syntactic, not behavioral, 2) datasets generally contain extreme exemplars producing a dataset in which it is easy to discriminate classes, and 3) the datasets provide significantly different representations of the data encountered in real-world systems. For ML to make more of an impact in the InfoSec community requires a change in the data (including the features and labels) that is used to bridge the current semantic gap. As a first step in enabling more behavioral analyses, we label existing malware datasets with behavioral features using open-source threat reports associated with malware families. This behavioral labeling alters the analysis from identifying intent (e.g. good vs bad) or malware family membership to an analysis of which behaviors are exhibited by an executable. We offer the annotations with the hope of inspiring future improvements in the data that will further bridge the semantic gap between the ML and InfoSec communities.
Google and Apple have jointly provided an API for exposure notification in order to implement decentralized contract tracing apps using Bluetooth Low Energy, the so-called "Google/Apple Proposal", which we abbreviate by "GAP". We demonstrate that in real-world scenarios the current GAP design is vulnerable to (i) profiling and possibly de-anonymizing infected persons, and (ii) relay-based wormhole attacks that basically can generate fake contacts with the potential of affecting the accuracy of an app-based contact tracing system. For both types of attack, we have built tools that can easily be…
Cybercrime is a complex phenomenon that spans both technical and human aspects. As such, two disjoint areas have been studying the problem from separate angles: the information security community and the environmental criminology one. Despite the large body of work produced by these communities in the past years, the two research efforts have largely remained disjoint, with researchers on one side not benefitting from the advancements proposed by the other. In this paper, we argue that it would be beneficial for the information security community to look at the theories and systematic framewor…
University of Liechtenstein（列支敦士登大学）；Robust Intelligence（鲁棒智能公司）；Norton Research Group（诺顿研究集团）；Meta；King’s College London（伦敦国王学院）
Recent years have seen a proliferation of research on adversarial machine learning. Numerous papers demonstrate powerful algorithmic attacks against a wide variety of machine learning (ML) models, and numerous other papers propose defenses that can withstand most attacks. However, abundant real-world evidence suggests that actual attackers use simple tactics to subvert ML-driven systems, and as a result security practitioners have not prioritized adversarial ML defenses. Motivated by the apparent gap between researchers and practitioners, this position paper aims to bridge the two domains. We …
Bridging Safety and Security in Complex Systems: A Model-Based Approach with SAFT-GT Toolchain
Universität Liechtenstein（列支敦士登大学）；Chalmers University of Technology and University of Gothenburg（查尔姆斯理工大学和哥德堡大学）；Ulm University（乌尔姆大学）；University of Innsbruck（因斯布鲁克大学）；German Aerospace Center (DLR)（德国航空航天中心）；University of Cologne（科隆大学）
In the last decade, deep learning algorithms have become very popular thanks to the achieved performance in many machine learning and computer vision tasks. However, most of the deep learning architectures are vulnerable to so called adversarial examples. This questions the security of deep neural networks (DNN) for many security- and trust-sensitive domains. The majority of the proposed existing adversarial attacks are based on the differentiability of the DNN cost function.Defence strategies are mostly based on machine learning and signal processing principles that either try to detect-rejec…
On-device machine learning (ML) is quickly gaining popularity among mobile apps. It allows offline model inference while preserving user privacy. However, ML models, considered as core intellectual properties of model owners, are now stored on billions of untrusted devices and subject to potential thefts. Leaked models can cause both severe financial loss and security consequences. This paper presents the first empirical study of ML model protection on mobile devices. Our study aims to answer three open questions with quantitative evidence: How widely is model protection used in apps? How robu…
There is a perceived disconnect between how ad hoc industry solutions and academic research solutions in cyber security are developed and applied. Is there a difference in philosophy in how solutions to cyber security problems are developed by industry and by academia. What could academia and industry do to bridge this gap and speed up the development and use of effective cybersecurity solutions? This paper provides an overview of the most critical gaps and solutions identified by an interdisciplinary expert exchange on the topic. The discussion was held in the form of the webinar "Bridging th…
In process algebras, security properties are expressed as equivalences between processes, but which equivalence is suitable is not clear. This means that there is a gap between an intuitive security notion and the formulation. Appropriate formalization is essential for verification, and our purpose is bridging this gap. By chasing scope extrusions, we prove that trace equivalence is congruent. Moreover, we construct an epistemic logic for the applied pi calculus and show that its logical equivalence agrees with the trace equivalence. We use the epistemic logic to show that trace equivalence is…
As digital technologies become more prevalent there is a growing awareness of the importance of good security and privacy practices. The tools and techniques used to achieve this are typically designed with the living user in mind, with little consideration of how they should or will perform after the user has died. We report on two workshops carried out with users of password managers to explore their views on the post-mortem sharing, security and privacy of a range of common digital assets. We discuss a post-mortem privacy paradox where users recognise value in planning for their digital leg…
This paper reviews existing Intrusion Detection Systems (IDS) that target the Mobile Cloud Computing (MCC), Cloud Computing (CC), and Mobile Device (MD) environment. The review identifies the drawbacks in existing solutions and proposes a novel approach towards enhancing the security of the User Layer (UL) in the MCC environment. The approach named MINDPRES (Mobile- Cloud Intrusion Detection and Prevention System) combines a host-based IDS and network-based IDS using Machine Learning (ML) techniques. It applies dynamic analysis of both device resources and network traffic in order to detect ma…
SAP Security Research, SAP Global Security（SAP全球安全事业部SAP安全研究院）；Hasso Plattner Institute, University of Potsdam（波茨坦大学哈索·普拉特纳研究所）；SAP Portfolio Strategy & Technology Adoption, SAP SE（SAP SE SAP产品组合战略与技术采纳部）；Logistic & IT Services, City of Antibes Juan-les-Pins（昂蒂布-朱安雷宾市物流与信息技术服务部）
Current developments in Enterprise Systems observe a paradigm shift, moving the needle from the backend to the edge sectors of those; by distributing data, decentralizing applications and integrating novel components seamlessly to the central systems. Distributively deployed AI capabilities will thrust this transition. Several non-functional requirements arise along with these developments, security being at the center of the discussions. Bearing those requirements in mind, hereby we propose an approach to holistically protect distributed Deep Neural Network (DNN) based/enhanced software asset…
Website fingerprinting (WF) attacks, usually conducted with the help of a machine learning-based classifier, enable a network eavesdropper to pinpoint which web page a user is accessing through the inspection of traffic patterns. These attacks have been shown to succeed even when users browse the Internet through encrypted tunnels, e.g., through Tor or VPNs. To assess the security of new defenses against WF attacks, recent works have proposed feature-dependent theoretical frameworks that estimate the Bayes error of an adversary's features set or the mutual information leaked by manually-crafte…
Development of new machine learning models is typically done on manually curated data sets, making them unsuitable for evaluating the models' performance during operations, where the evaluation needs to be performed automatically on incoming streams of new data. Unfortunately, pure reliance on a fully automatic pipeline for monitoring model performance makes it difficult to understand if any observed performance issues are due to model performance, pipeline issues, emerging data distribution biases, or some combination of the above. With this in mind, we developed a web-based visualization sys…
Modern corporations physically separate their sensitive computational infrastructure from public or other accessible networks in order to prevent cyber-attacks. However, attackers still manage to infect these networks, either by means of an insider or by infiltrating the supply chain. Therefore, an attacker's main challenge is to determine a way to command and control the compromised hosts that are isolated from an accessible network (e.g., the Internet). In this paper, we propose a new adversarial model that shows how an air gapped network can receive communications over a covert thermal chan…
Blockchain Infrastructure for Intelligent Cyber--Physical--Social Systems:Post-Quantum Security, Interoperability, and Trustworthy Data Economies in the Era of Embodied AI
Hong Kong University of Science and Technology（香港理工大学）；Sun Yat-sen University（中山大学）；Amazon Web Services（亚马逊网络服务）；Duke Kunshan University（杜克昆山大学）
The increasing digitization and interconnection of legacy Industrial Control Systems (ICSs) open new vulnerability surfaces, exposing such systems to malicious attackers. Furthermore, since ICSs are often employed in critical infrastructures (e.g., nuclear plants) and manufacturing companies (e.g., chemical industries), attacks can lead to devastating physical damages. In dealing with this security requirement, the research community focuses on developing new security mechanisms such as Intrusion Detection Systems (IDSs), facilitated by leveraging modern machine learning techniques. However, t…
Online programming discussion platforms such as Stack Overflow serve as a rich source of information for software developers. Available information include vibrant discussions and oftentimes ready-to-use code snippets. Anecdotes report that software developers copy and paste code snippets from those information sources for convenience reasons. Such behavior results in a constant flow of community-provided code snippets into production software. To date, the impact of this behaviour on code security is unknown. We answer this highly important question by quantifying the proliferation of securit…
Tsinghua University（清华大学）；Beijing National Research Center for Information Science and Technology（北京信息科学与技术国家研究中心）；Institute for Artificial Intelligence, Tsinghua University（清华大学人工智能研究院）；Huazhong University of Science and Technology（华中科技大学）；Beihang University（北京航空航天大学）
Adversarial attacks and backdoor attacks are two common security threats that hang over deep learning. Both of them harness task-irrelevant features of data in their implementation. Text style is a feature that is naturally irrelevant to most NLP tasks, and thus suitable for adversarial and backdoor attacks. In this paper, we make the first attempt to conduct adversarial and backdoor attacks based on text style transfer, which is aimed at altering the style of a sentence while preserving its meaning. We design an adversarial attack method and a backdoor attack method, and conduct extensive exp…
Security still remains an afterthought in modern Electronic Design Automation (EDA) tools, which solely focus on enhancing performance and reducing the chip size. Typically, the security analysis is conducted by hand, leading to vulnerabilities in the design remaining unnoticed. Security-aware EDA tools assist the designer in the identification and removal of security threats while keeping performance and area in mind. State-of-the-art approaches utilize information flow analysis to spot unintended information leakages in design structures. However, the classification of such threats is binary…
MoRSE: Bridging the Gap in Cybersecurity Expertise with Retrieval Augmented Generation
Istituto di Informatica e Telematica, Consiglio Nazionale Delle Ricerche（国家研究委员会信息与自动化研究所）；TeCIP, Scuola Universitaria Superiore Sant’Anna（圣安娜高等学校TeCIP研究所）；University of Padua（帕多瓦大学）；Delft University of Technology（代尔夫特理工大学）
Although almost all information about Smart Contract addresses is shared via websites, emails, or other forms of digital communication, Blockchains and distributed ledger technology are unable to establish secure bindings between websites and corresponding Smart Contracts. For a user, it is impossible to differentiate whether a website links to a legitimate Smart Contract set up by owners of a business or to an illicit contract aiming to steal users' funds. Surprisingly, current attempts to solve this issue mostly comprise of information redundancy, e.g., displaying contract addresses multiple…
