The training and creation of deep learning model is usually costly, thus it can be regarded as an intellectual property (IP) of the model creator. However, malicious users who obtain high-performance models may illegally copy, redistribute, or abuse the models without permission. To deal with such security threats, a few deep neural networks (DNN) IP protection methods have been proposed in recent years. This paper attempts to provide a review of the existing DNN IP protection works and also an outlook. First, we propose the first taxonomy for DNN IP protection methods in terms of six attributes: scenario, mechanism, capacity, type, function, and target models. Then, we present a survey on existing DNN IP protection works in terms of the above six attributes, especially focusing on the challenges these methods face, whether these methods can provide proactive protection, and their resistances to different levels of attacks. After that, we analyze the potential attacks on DNN IP protection methods from the aspects of model modifications, evasion attacks, and active attacks. Besides, a systematic evaluation method for DNN IP protection methods with respect to basic functional metrics, attack-resistance metrics, and customized metrics for different application scenarios is given. Lastly, future research opportunities and challenges on DNN IP protection are presented.
Since training a deep neural network (DNN) is costly, the well-trained deep models can be regarded as valuable intellectual property (IP) assets. The IP protection associated with deep models has been receiving increasing attentions in recent years. Passport-based method, which replaces normalization layers with passport layers, has been one of the few protection solutions that are claimed to be secure against advanced attacks. In this work, we tackle the issue of evaluating the security of passport-based IP protection methods. We propose a novel and effective ambiguity attack against passport…
Capitalise on deep learning models, offering Natural Language Processing (NLP) solutions as a part of the Machine Learning as a Service (MLaaS) has generated handsome revenues. At the same time, it is known that the creation of these lucrative deep models is non-trivial. Therefore, protecting these inventions intellectual property rights (IPR) from being abused, stolen and plagiarized is vital. This paper proposes a practical approach for the IPR protection on recurrent neural networks (RNN) without all the bells and whistles of existing IPR solutions. Particularly, we introduce the Gatekeeper…
DNF: Dual-Layer Nested Fingerprinting for Large Language Model Intellectual Property Protection
Zhejiang University（浙江大学）；Binjiang Institute of Zhejiang University（浙江大学滨江学院）；Hangzhou Dianzi University（杭州电子科技大学）；Hefei University of Technology（合肥工业大学）
Intellectual Property Protection for 3D Gaussian Splatting Assets: A Survey
Sydney AI Centre, The University of Sydney（悉尼人工智能中心，悉尼大学）；The University of Melbourne（墨尔本大学）；Mohamed bin Zayed University of Artificial Intelligence（莫莫迪·本·扎耶德人工智能大学）
Previous works have validated that text generation APIs can be stolen through imitation attacks, causing IP violations. In order to protect the IP of text generation APIs, a recent work has introduced a watermarking algorithm and utilized the null-hypothesis test as a post-hoc ownership verification on the imitation models. However, we find that it is possible to detect those watermarks via sufficient statistics of the frequencies of candidate watermarking words. To address this drawback, in this paper, we propose a novel Conditional wATERmarking framework (CATER) for protecting the IP of text…
The data is an important asset of an organization and it is essential to keep this asset secure. It requires security in whatever state is it i.e. data at rest, data in use, and data in transit. There is a need to pay more attention to it when the third party is included i.e. when the data is stored in the cloud then it requires more security. Since confidential data can reside on a variety of computing devices (physical servers, virtual servers, databases, file servers, PCs, point-of-sale devices, flash drives, and mobile devices) and move through a variety of network access points (wireline,…
SAP Security Research, SAP Global Security（SAP全球安全事业部SAP安全研究院）；Hasso Plattner Institute, University of Potsdam（波茨坦大学哈索·普拉特纳研究所）；SAP Portfolio Strategy & Technology Adoption, SAP SE（SAP SE SAP产品组合战略与技术采纳部）；Logistic & IT Services, City of Antibes Juan-les-Pins（昂蒂布-朱安雷宾市物流与信息技术服务部）
Current developments in Enterprise Systems observe a paradigm shift, moving the needle from the backend to the edge sectors of those; by distributing data, decentralizing applications and integrating novel components seamlessly to the central systems. Distributively deployed AI capabilities will thrust this transition. Several non-functional requirements arise along with these developments, security being at the center of the discussions. Bearing those requirements in mind, hereby we propose an approach to holistically protect distributed Deep Neural Network (DNN) based/enhanced software asset…
In order to prevent deep neural networks from being infringed by unauthorized parties, we propose a generic solution which embeds a designated digital passport into a network, and subsequently, either paralyzes the network functionalities for unauthorized usages or maintain its functionalities in the presence of a verified passport. Such a desired network behavior is successfully demonstrated in a number of implementation schemes, which provide reliable, preventive and timely protections against tens of thousands of fake-passport deceptions. Extensive experiments also show that the deep neural…
Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks
SBA Research（SBA研究中心）；Vienna University of Technology（维也纳工业大学）
In the domain of medical image processing, medical device manufacturers protect their intellectual property in many cases by shipping only compiled software, i.e. binary code which can be executed but is difficult to be understood by a potential attacker. In this paper, we investigate how well this procedure is able to protect image processing algorithms. In particular, we investigate whether the computation of mono-energetic images and iodine maps from dual energy CT data can be reverse-engineered by machine learning methods. Our results indicate that both can be approximated using only one s…
THEMIS: Towards Practical Intellectual Property Protection for Post-Deployment On-Device Deep Learning Models
The University of Melbourne（墨尔本大学）；The University of Western Australia（西澳大学）；City University of Hong Kong（香港城市大学）；Technical University of Munich（慕尼黑工业大学）
University of Malaya（马来亚大学）；WeBank（微众银行）；Hong Kong University of Science and Technology（香港科技大学）
Ever since Machine Learning as a Service (MLaaS) emerges as a viable business that utilizes deep learning models to generate lucrative revenue, Intellectual Property Right (IPR) has become a major concern because these deep learning models can easily be replicated, shared, and re-distributed by any unauthorized third parties. To the best of our knowledge, one of the prominent deep learning models - Generative Adversarial Networks (GANs) which has been widely used to create photorealistic image are totally unprotected despite the existence of pioneering IPR protection methodology for Convolutio…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；City University of Hong Kong（香港城市大学）；Beijing Electronic Science and Technology Institute（北京电子科技学院）；Wormpex AI Research LLC（Wormpex AI研究有限责任公司）
Despite the tremendous success, deep neural networks are exposed to serious IP infringement risks. Given a target deep model, if the attacker knows its full information, it can be easily stolen by fine-tuning. Even if only its output is accessible, a surrogate model can be trained through student-teacher learning by generating many input-output training pairs. Therefore, deep model IP protection is important and necessary. However, it is still seriously under-researched. In this work, we propose a new model watermarking framework for protecting deep networks trained for low-level computer visi…
Intellectual Property Protection for Deep Learning Model and Dataset Intelligence
Nanjing University of Science and Technology（南京理工大学）；State Key Laboratory of Integrated Services Networks (Xidian University)（综合业务网国家重点实验室（西安电子科技大学））；The University of Western Australia（西澳大学）；Zhejiang University（浙江大学）；University of Newcastle（纽卡斯尔大学）；University of Wollongong（卧龙岗大学）
The increasing cost of integrated circuit (IC) fabrication has driven most companies to "go fabless" over time. The corresponding outsourcing trend gave rise to various attack vectors, e.g., illegal overproduction of ICs, piracy of the design intellectual property (IP), or insertion of hardware Trojans (HTs). These attacks are possibly conducted by untrusted entities residing all over the supply chain, ranging from untrusted foundries, test facilities, even to end-users. To overcome this multitude of threats, various techniques have been proposed over the past decade. In this paper, we review …
With the broad application of deep neural networks, the necessity of protecting them as intellectual properties has become evident. Numerous watermarking schemes have been proposed to identify the owner of a deep neural network and verify the ownership. However, most of them focused on the watermark embedding rather than the protocol for provable verification. To bridge the gap between those proposals and real-world demands, we study the deep learning model intellectual property protection in three scenarios: the ownership proof, the federated learning, and the intellectual property transfer. …
Code obfuscation is a major tool for protecting software intellectual property from attacks such as reverse engineering or code tampering. Yet, recently proposed (automated) attacks based on Dynamic Symbolic Execution (DSE) shows very promising results, hence threatening software integrity. Current defenses are not fully satisfactory, being either not efficient against symbolic reasoning, or affecting runtime performance too much, or being too easy to spot. We present and study a new class of anti-DSE protections coined as path-oriented protections targeting the weakest spot of DSE, namely pat…
PragLocker: Protecting Agent Intellectual Property in Untrusted Deployments via Non-Portable Prompts
Zhejiang University（浙江大学）；Management Center, School of Software Technology (Ningbo), Zhejiang University（浙江大学软件学院（宁波）管理中心）；University of Science and Technology of China（中国科学技术大学）；Chang'an University（长安大学）；Washington University in St. Louis（圣路易斯华盛顿大学）
In this work, we propose LUT-Lock, a novel Look-Up-Table-based netlist obfuscation algorithm, for protecting the intellectual property that is mapped to an FPGA bitstream or an ASIC netlist. We, first, illustrate the effectiveness of several key features that make the LUT-based obfuscation more resilient against SAT attacks and then we embed the proposed key features into our proposed LUT-Lock algorithm. We illustrates that LUT-Lock maximizes the resiliency of the LUT-based obfuscation against SAT attacks by forcing a near exponential increase in the execution time of a SAT solver with respect…
Secure and trustworthy execution in heterogeneous SoCs is a major priority in the modern computing system. Security of SoCs mainly addresses two broad layers of trust issues: 1. Protection against hardware security threats(Side-channel, IP Privacy, Cloning, Fault Injection, and Denial of Service); and 2. Protection against malicious software attacks running on SoC processors. To resist malicious software-level attackers from gaining unauthorized access and compromising security, we propose a root of trust-based trusted execution mechanism \textbf{\textit{(named as \textbf{TrustToken}) }}. Trus…
Intellectual Property in Graph-Based Machine Learning as a Service: Attacks and Defenses
Department of Computer Science, Florida State University（佛罗里达州立大学计算机科学系）；Northeastern University（东北大学）；University of Wisconsin-Madison（威斯康星大学麦迪逊分校）；University of Notre Dame（诺丁汉大学）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
Existing logic-locking attacks are known to successfully decrypt functionally correct key of a locked combinational circuit. It is possible to extend these attacks to real-world Silicon-based Intellectual Properties (IPs, which are sequential circuits) through scan-chains by selectively initializing the combinational logic and analyzing the responses. In this paper, we propose SeqL, which achieves functional isolation and locks selective flip-flop functional-input/scan-output pairs, thus rendering the decrypted key functionally incorrect. We conduct a formal study of the scan-locking problem a…
Increased connectivity has made us all more vulnerable. Cyberspace, besides all its benefits, spawned more devices to hack and more opportunities to commit cybercrime. Criminals have found it lucrative to target both individuals and businesses, by holding or stealing their assets via different types of cyber attacks. The cyber-enabled theft of Intellectual Property (IP), as one of the most important and critical intangible assets of nations, organizations and individuals, by foreign countries has been a devastating challenge of the United States (U.S.) in the past decades. In this study, we co…
Embedded devices in the Internet of Things (IoT) face a wide variety of security challenges. For example, software attackers perform code injection and code-reuse attacks on their remote interfaces, and physical access to IoT devices allows to tamper with code in memory, steal confidential Intellectual Property (IP), or mount fault attacks to manipulate a CPU's control flow. In this work, we present Sponge-based Control Flow Protection (SCFP). SCFP is a stateful, sponge-based scheme to ensure the confidentiality of software IP and its authentic execution on IoT devices. At compile time, SCFP e…
University of New South Wales（新南威尔士大学）；The Australian National University（澳大利亚国立大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Generative AI (e.g., Generative Adversarial Networks - GANs) has become increasingly popular in recent years. However, Generative AI introduces significant concerns regarding the protection of Intellectual Property Rights (IPR) (resp. model accountability) pertaining to images (resp. toxic images) and models (resp. poisoned models) generated. In this paper, we propose an evaluation framework to provide a comprehensive overview of the current state of the copyright protection measures for GANs, evaluate their performance across a diverse range of GAN architectures, and identify the factors that…
Cloud computing opportunities have presented service options for users that are both economical and flexible to use requirements. However, the risk analysis for the user identifies vulnerabilities for intellectual property ownership and vulnerabilities for the identification of rightful property owners when cloud services are used. It is common for image owners to embed watermarks and other security mechanisms into their property so that the rightful ownership may be identified. In this paper we present a design that overcomes many of the current limitations in cloud watermarking uses and prop…
Deep Learning (DL) models have caused a paradigm shift in our ability to comprehend raw data in various important fields, ranging from intelligence warfare and healthcare to autonomous transportation and automated manufacturing. A practical concern, in the rush to adopt DL models as a service, is protecting the models against Intellectual Property (IP) infringement. The DL models are commonly built by allocating significant computational resources that process vast amounts of proprietary training data. The resulting models are therefore considered to be the IP of the model builder and need to …
Monash University（蒙纳士大学）；The Australian National University（澳大利亚国立大学）；Sony AI（索尼人工智能）；Microsoft Research Asia（微软亚洲研究院）；UC Berkeley（加州大学伯克利分校）
Nowadays, due to the breakthrough in natural language generation (NLG), including machine translation, document summarization, image captioning, etc NLG models have been encapsulated in cloud APIs to serve over half a billion people worldwide and process over one hundred billion word generations per day. Thus, NLG APIs have already become essential profitable services in many commercial companies. Due to the substantial financial and intellectual investments, service providers adopt a pay-as-you-use policy to promote sustainable market growth. However, recent works have shown that cloud platfo…
Extracting the architecture of layers of a given deep neural network (DNN) through hardware-based side channels allows adversaries to steal its intellectual property and even launch powerful adversarial attacks on the target system. In this work, we propose DNN-Alias, an obfuscation method for DNNs that forces all the layers in a given network to have similar execution traces, preventing attack models from differentiating between the layers. Towards this, DNN-Alias performs various layer-obfuscation operations, e.g., layer branching, layer deepening, etc, to alter the run-time traces while mai…
Internet of Things (IoT) devices are becoming ubiquitous in our lives, with applications spanning from the consumer domain to commercial and industrial systems. The steep growth and vast adoption of IoT devices reinforce the importance of sound and robust cybersecurity practices during the device development life-cycles. IoT-related vulnerabilities, if successfully exploited can affect, not only the device itself, but also the application field in which the IoT device operates. Evidently, identifying and addressing every single vulnerability is an arduous, if not impossible, task. Attack taxon…
