Deep Neural Network (DNN) classifiers are known to be vulnerable to Trojan or backdoor attacks, where the classifier is manipulated such that it misclassifies any input containing an attacker-determined Trojan trigger. Backdoors compromise a model's integrity, thereby posing a severe threat to the landscape of DNN-based classification. While multiple defenses against such attacks exist for classifiers in the image domain, there have been limited efforts to protect classifiers in the text domain.
We present Trojan-Miner (T-Miner) -- a defense framework for Trojan attacks on DNN-based text classifiers. T-Miner employs a sequence-to-sequence (seq-2-seq) generative model that probes the suspicious classifier and learns to produce text sequences that are likely to contain the Trojan trigger. T-Miner then analyzes the text produced by the generative model to determine if they contain trigger phrases, and correspondingly, whether the tested classifier has a backdoor. T-Miner requires no access to the training dataset or clean inputs of the suspicious classifier, and instead uses synthetically crafted "nonsensical" text inputs to train the generative model. We extensively evaluate T-Miner on 1100 model instances spanning 3 ubiquitous DNN model architectures, 5 different classification tasks, and a variety of trigger phrases. We show that T-Miner detects Trojan and clean models with a 98.75% overall accuracy, while achieving low false positives on clean models. We also show that T-Miner is robust against a variety of targeted, advanced attacks from an adaptive attacker.
Zhejiang University（浙江大学）；Pennsylvania State University（宾夕法尼亚州立大学）；Chinese Aeronautical Establishment（中国航空研究院）；National University of Defense Technology（国防科技大学）
Trojan attack on deep neural networks, also known as backdoor attack, is a typical threat to artificial intelligence. A trojaned neural network behaves normally with clean inputs. However, if the input contains a particular trigger, the trojaned model will have attacker-chosen abnormal behavior. Although many backdoor detection methods exist, most of them assume that the defender has access to a set of clean validation samples or samples with the trigger, which may not hold in some crucial real-world cases, e.g., the case where the defender is the maintainer of model-sharing platforms. Thus, i…
This paper defends the design of hardware Trojan proposed in the paper "Defeating HaTCh: Building Malicious IP Cores" by defining the four critical properties which are required to be satisfied to classify it to belong to class of Deterministic Hardware Trojan. This is to counter the claims of authors of HaTCh in their paper "Comments on Defeating HaTCh".
Deep Neural Networks are vulnerable to Trojan (or backdoor) attacks. Reverse-engineering methods can reconstruct the trigger and thus identify affected models. Existing reverse-engineering methods only consider input space constraints, e.g., trigger size in the input space. Expressly, they assume the triggers are static patterns in the input space and fail to detect models with feature space triggers such as image style transformations. We observe that both input-space and feature-space Trojans are associated with feature space hyperplanes. Based on this observation, we design a novel reverse-…
Applied Artificial Intelligence Institute, Deakin University（迪肯大学应用人工智能研究所）
In this paper, we propose a novel host-free Trojan attack with triggers that are fixed in the semantic space but not necessarily in the pixel space. In contrast to existing Trojan attacks which use clean input images as hosts to carry small, meaningless trigger patterns, our attack considers triggers as full-sized images belonging to a semantically meaningful object class. Since in our attack, the backdoored classifier is encouraged to memorize the abstract semantics of the trigger images than any specific fixed pattern, it can be later triggered by semantically similar but different looking i…
We propose Februus; a new idea to neutralize highly potent and insidious Trojan attacks on Deep Neural Network (DNN) systems at run-time. In Trojan attacks, an adversary activates a backdoor crafted in a deep neural network model using a secret trigger, a Trojan, applied to any input to alter the model's decision to a target prediction---a target determined by and only known to the attacker. Februus sanitizes the incoming input by surgically removing the potential trigger artifacts and restoring the input for the classification task. Februus enables effective Trojan mitigation by sanitizing in…
Since the inception of the Integrated Circuit (IC), the size of the transistors used to construct them has continually shrunk. While this advancement significantly improves computing capability, fabrication costs have skyrocketed. As a result, most IC designers must now outsource fabrication. Outsourcing, however, presents a security threat: comprehensive post-fabrication inspection is infeasible given the size of modern ICs, so it is nearly impossible to know if the foundry has altered the original design during fabrication (i.e., inserted a hardware Trojan). Defending against a foundry-side …
New Jersey Institute of Technology（新泽西理工学院）；Jordan University of Science & Technology（约旦科技大学）；Qatar Computing Research Institute（卡塔尔计算研究所）；HBKU（哈马德·本·哈利法大学）
Trojan backdoor is a poisoning attack against Neural Network (NN) classifiers in which adversaries try to exploit the (highly desirable) model reuse property to implant Trojans into model parameters for backdoor breaches through a poisoned training process. Most of the proposed defenses against Trojan attacks assume a white-box setup, in which the defender either has access to the inner state of NN or is able to run back-propagation through it. In this work, we propose a more practical black-box defense, dubbed TrojDef, which can only run forward-pass of the NN. TrojDef tries to identify and f…
We have witnessed the continuing arms race between backdoor attacks and the corresponding defense strategies on Deep Neural Networks (DNNs). Most state-of-the-art defenses rely on the statistical sanitization of the "inputs" or "latent DNN representations" to capture trojan behaviour. In this paper, we first challenge the robustness of such recently reported defenses by introducing a novel variant of targeted backdoor attack, called "low-confidence backdoor attack". We also propose a novel defense technique, called "HaS-Nets". "Low-confidence backdoor attack" exploits the confidence labels ass…
Recent work has demonstrated robust mechanisms by which attacks can be orchestrated on machine learning models. In contrast to adversarial examples, backdoor or trojan attacks embed surgically modified samples with targeted labels in the model training process to cause the targeted model to learn to misclassify chosen samples in the presence of specific triggers, while keeping the model performance stable across other nominal samples. However, current published research on trojan attacks mainly focuses on classification problems, which ignores sequential dependency between inputs. In this pape…
As cross-chain technologies make the interactions among different blockchains (hereinafter "chains") possible, multi-chains consensus is becoming more and more important in blockchain networks. However, more attention has been paid to the single-chain consensus schemes. The multi-chains consensus with trusted miners participation has been not considered, thus offering opportunities for malicious users to launch Diverse Miners Behaviors (DMB) attacks on different chains. DMB attackers can be friendly in the consensus process of some chains called mask-chains to enhance trust value, while on oth…
Outsourced deep neural networks have been demonstrated to suffer from patch-based trojan attacks, in which an adversary poisons the training sets to inject a backdoor in the obtained model so that regular inputs can be still labeled correctly while those carrying a specific trigger are falsely given a target label. Due to the severity of such attacks, many backdoor detection and containment systems have recently, been proposed for deep neural networks. One major category among them are various model inspection schemes, which hope to detect backdoors before deploying models from non-trusted thi…
Backdoor (Trojan) attacks are emerging threats against deep neural networks (DNN). A DNN being attacked will predict to an attacker-desired target class whenever a test sample from any source class is embedded with a backdoor pattern; while correctly classifying clean (attack-free) test samples. Existing backdoor defenses have shown success in detecting whether a DNN is attacked and in reverse-engineering the backdoor pattern in a "post-training" regime: the defender has access to the DNN to be inspected and a small, clean dataset collected independently, but has no access to the (possibly poi…
We revisit the fundamental question of Bitcoin's security against double spending attacks. While previous work has bounded the probability that a transaction is reversed, we show that no such guarantee can be effectively given if the attacker can choose when to launch the attack. Other approaches that bound the cost of an attack have erred in considering only limited attack scenarios, and in fact it is easy to show that attacks may not cost the attacker at all. We therefore provide a different interpretation of the results presented in previous papers and correct them in several ways. We provi…
The backdoor or Trojan attack is a severe threat to deep neural networks (DNNs). Researchers find that DNNs trained on benign data and settings can also learn backdoor behaviors, which is known as the natural backdoor. Existing works on anti-backdoor learning are based on weak observations that the backdoor and benign behaviors can differentiate during training. An adaptive attack with slow poisoning can bypass such defenses. Moreover, these methods cannot defend natural backdoors. We found the fundamental differences between backdoor-related neurons and benign neurons: backdoor-related neuron…
Split manufacturing is a promising technique to defend against fab-based malicious activities such as IP piracy, overbuilding, and insertion of hardware Trojans. However, a network flow-based proximity attack, proposed by Wang et al. (DAC'16) [1], has demonstrated that most prior art on split manufacturing is highly vulnerable. Here in this work, we present two practical layout techniques towards secure split manufacturing: (i) gate-level graph coloring and (ii) clustering of same-type gates. Our approach shows promising results against the advanced proximity attack, lowering its success rate …
The transistors used to construct Integrated Circuits (ICs) continue to shrink. While this shrinkage improves performance and density, it also reduces trust: the price to build leading-edge fabrication facilities has skyrocketed, forcing even nation states to outsource the fabrication of high-performance ICs. Outsourcing fabrication presents a security threat because the black-box nature of a fabricated IC makes comprehensive inspection infeasible. Since prior work shows the feasibility of fabrication-time attackers' evasion of existing post-fabrication defenses, IC designers must be able to p…
