Diffusion models have rapidly become a vital part of deep generative architectures, given today's increasing demands. Obtaining large, high-performance diffusion models demands significant resources, highlighting their importance as intellectual property worth protecting. However, existing watermarking techniques for ownership verification are insufficient when applied to diffusion models. Very recent research in watermarking diffusion models either exposes watermarks during task generation, which harms the imperceptibility, or is developed for conditional diffusion models that require prompts to trigger the watermark. This paper introduces WDM, a novel watermarking solution for diffusion models without imprinting the watermark during task generation. It involves training a model to concurrently learn a Watermark Diffusion Process (WDP) for embedding watermarks alongside the standard diffusion process for task generation. We provide a detailed theoretical analysis of WDP training and sampling, relating it to a shifted Gaussian diffusion process via the same reverse noise. Extensive experiments are conducted to validate the effectiveness and robustness of our approach in various trigger and watermark data configurations.
University of New South Wales（新南威尔士大学）；The Australian National University（澳大利亚国立大学）；CSIRO’s Data61（澳大利亚联邦科学与工业研究组织Data61研究所）
Generative AI (e.g., Generative Adversarial Networks - GANs) has become increasingly popular in recent years. However, Generative AI introduces significant concerns regarding the protection of Intellectual Property Rights (IPR) (resp. model accountability) pertaining to images (resp. toxic images) and models (resp. poisoned models) generated. In this paper, we propose an evaluation framework to provide a comprehensive overview of the current state of the copyright protection measures for GANs, evaluate their performance across a diverse range of GAN architectures, and identify the factors that…
Shandong University（山东大学）；CISPA Helmholtz Center for Information Security（CISPA亥姆霍兹信息安全中心）
Deep learning techniques have made tremendous progress in a variety of challenging tasks, such as image recognition and machine translation, during the past decade. Training deep neural networks is computationally expensive and requires both human and intellectual resources. Therefore, it is necessary to protect the intellectual property of the model and externally verify the ownership of the model. However, previous studies either fail to defend against the evasion attack or have not explicitly dealt with fraudulent claims of ownership by adversaries. Furthermore, they can not establish a cle…
Previous works have validated that text generation APIs can be stolen through imitation attacks, causing IP violations. In order to protect the IP of text generation APIs, a recent work has introduced a watermarking algorithm and utilized the null-hypothesis test as a post-hoc ownership verification on the imitation models. However, we find that it is possible to detect those watermarks via sufficient statistics of the frequencies of candidate watermarking words. To address this drawback, in this paper, we propose a novel Conditional wATERmarking framework (CATER) for protecting the IP of text…
A deep neural network (DNN) classifier represents a model owner's intellectual property as training a DNN classifier often requires lots of resource. Watermarking was recently proposed to protect the intellectual property of DNN classifiers. However, watermarking suffers from a key limitation: it sacrifices the utility/accuracy of the model owner's classifier because it tampers the classifier's training or fine-tuning process. In this work, we propose IPGuard, the first method to protect intellectual property of DNN classifiers that provably incurs no accuracy loss for the classifiers. Our key…
Monash University（蒙纳士大学）；The Australian National University（澳大利亚国立大学）；Sony AI（索尼人工智能）；Microsoft Research Asia（微软亚洲研究院）；UC Berkeley（加州大学伯克利分校）
Nowadays, due to the breakthrough in natural language generation (NLG), including machine translation, document summarization, image captioning, etc NLG models have been encapsulated in cloud APIs to serve over half a billion people worldwide and process over one hundred billion word generations per day. Thus, NLG APIs have already become essential profitable services in many commercial companies. Due to the substantial financial and intellectual investments, service providers adopt a pay-as-you-use policy to promote sustainable market growth. However, recent works have shown that cloud platfo…
Identifying Appropriate Intellectual Property Protection Mechanisms for Machine Learning Models: A Systematization of Watermarking, Fingerprinting, Model Access, and Attacks
SBA Research（SBA研究中心）；Vienna University of Technology（维也纳工业大学）
Aggressive time-to-market constraints and enormous hardware design and fabrication costs have pushed the semiconductor industry toward hardware Intellectual Properties (IP) core design. However, the globalization of the integrated circuits (IC) supply chain exposes IP providers to theft and illegal redistribution of IPs. Watermarking and fingerprinting are proposed to detect IP piracy. Nevertheless, they come with additional hardware overhead and cannot guarantee IP security as advanced attacks are reported to remove the watermark, forge, or bypass it. In this work, we propose a novel methodol…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；City University of Hong Kong（香港城市大学）；Beijing Electronic Science and Technology Institute（北京电子科技学院）；Wormpex AI Research LLC（Wormpex AI研究有限责任公司）
Despite the tremendous success, deep neural networks are exposed to serious IP infringement risks. Given a target deep model, if the attacker knows its full information, it can be easily stolen by fine-tuning. Even if only its output is accessible, a surrogate model can be trained through student-teacher learning by generating many input-output training pairs. Therefore, deep model IP protection is important and necessary. However, it is still seriously under-researched. In this work, we propose a new model watermarking framework for protecting deep networks trained for low-level computer visi…
With the increasing application value of machine learning, the intellectual property (IP) rights of deep neural networks (DNN) are getting more and more attention. With our analysis, most of the existing DNN watermarking methods can resist fine-tuning and pruning attack, but distillation attack. To address these problem, we propose a new DNN watermarking framework, Unified Soft-label Perturbation (USP), having a detector paired with the model to be watermarked, and Customized Soft-label Perturbation (CSP), embedding watermark via adding perturbation into the model output probability distributi…
With the broad application of deep neural networks, the necessity of protecting them as intellectual properties has become evident. Numerous watermarking schemes have been proposed to identify the owner of a deep neural network and verify the ownership. However, most of them focused on the watermark embedding rather than the protocol for provable verification. To bridge the gap between those proposals and real-world demands, we study the deep learning model intellectual property protection in three scenarios: the ownership proof, the federated learning, and the intellectual property transfer. …
SAP Security Research, SAP Global Security（SAP全球安全事业部SAP安全研究院）；Hasso Plattner Institute, University of Potsdam（波茨坦大学哈索·普拉特纳研究所）；SAP Portfolio Strategy & Technology Adoption, SAP SE（SAP SE SAP产品组合战略与技术采纳部）；Logistic & IT Services, City of Antibes Juan-les-Pins（昂蒂布-朱安雷宾市物流与信息技术服务部）
Current developments in Enterprise Systems observe a paradigm shift, moving the needle from the backend to the edge sectors of those; by distributing data, decentralizing applications and integrating novel components seamlessly to the central systems. Distributively deployed AI capabilities will thrust this transition. Several non-functional requirements arise along with these developments, security being at the center of the discussions. Bearing those requirements in mind, hereby we propose an approach to holistically protect distributed Deep Neural Network (DNN) based/enhanced software asset…
Digital watermarking system is a paramount for safeguarding valuable resources and information. Digital watermarks are generally imperceptible to the human eye and ear. Digital watermark can be used in video, audio and digital images for a wide variety of applications such as copy prevention right management, authentication and filtering of internet content. The proposed system is able to protect copyright or owner identification of digital media, such as audio, image, video, or text. The system permutated the watermark and embed the permutated watermark into the wavelet coefficients of the or…
Graph Neural Networks (GNNs) are valuable intellectual property, yet many watermarks rely on backdoor triggers that break under common model edits and create ownership ambiguity. We present InvGNN-WM, which ties ownership to a model's implicit perception of a graph invariant, enabling trigger-free, black-box verification with negligible task impact. A lightweight head predicts normalized algebraic connectivity on an owner-private carrier set; a sign-sensitive decoder outputs bits, and a calibrated threshold controls the false-positive rate. Across diverse node and graph classification datasets…
The commercialization of deep learning creates a compelling need for intellectual property (IP) protection. Deep neural network (DNN) watermarking has been proposed as a promising tool to help model owners prove ownership and fight piracy. A popular approach of watermarking is to train a DNN to recognize images with certain \textit{trigger} patterns. In this paper, we propose a novel evolutionary algorithm-based method to generate and optimize trigger patterns. Our method brings a siginificant reduction in false positive rates, leading to compelling proof of ownership. At the same time, it mai…
In this paper a new approach to image watermarking in wavelet domain is presented. The idea is to hide the watermark data in blocks of the block segmented image. Two schemes are presented based on this idea by embedding the watermark data in the low pass wavelet coefficients of each block. Due to low computational complexity of the proposed approach, this algorithm can be implemented in real time. Experimental results demonstrate the impercepti-bility of the proposed method and its high robustness against various attacks such as filtering, JPEG compres-sion, cropping, noise addition and geomet…
Peking University（北京大学）；Central South University（中南大学）；Fuzhou University（福州大学）；Jinan University（暨南大学）；Loughborough University（拉夫堡大学）
Federated learning (FL) allows multiple participants to collaboratively build deep learning (DL) models without directly sharing data. Consequently, the issue of copyright protection in FL becomes important since unreliable participants may gain access to the jointly trained model. Application of homomorphic encryption (HE) in secure FL framework prevents the central server from accessing plaintext models. Thus, it is no longer feasible to embed the watermark at the central server using existing watermarking schemes. In this paper, we propose a novel client-side FL watermarking scheme to tackl…
Since training a deep neural network (DNN) is costly, the well-trained deep models can be regarded as valuable intellectual property (IP) assets. The IP protection associated with deep models has been receiving increasing attentions in recent years. Passport-based method, which replaces normalization layers with passport layers, has been one of the few protection solutions that are claimed to be secure against advanced attacks. In this work, we tackle the issue of evaluating the security of passport-based IP protection methods. We propose a novel and effective ambiguity attack against passport…
The training and creation of deep learning model is usually costly, thus it can be regarded as an intellectual property (IP) of the model creator. However, malicious users who obtain high-performance models may illegally copy, redistribute, or abuse the models without permission. To deal with such security threats, a few deep neural networks (DNN) IP protection methods have been proposed in recent years. This paper attempts to provide a review of the existing DNN IP protection works and also an outlook. First, we propose the first taxonomy for DNN IP protection methods in terms of six attribut…
Centre Inria de l’Université de Rennes（雷恩大学法国国家信息与自动化研究所中心）
It is crucial to protect the intellectual property rights of DNN models prior to their deployment. The DNN should perform two main tasks: its primary task and watermarking task. This paper proposes a lightweight, reliable, and secure DNN watermarking that attempts to establish strong ties between these two tasks. The samples triggering the watermarking task are generated using image Mixup either from training or testing samples. This means that there is an infinity of triggers not limited to the samples used to embed the watermark in the model at training. The extensive experiments on image cl…
In order to prevent deep neural networks from being infringed by unauthorized parties, we propose a generic solution which embeds a designated digital passport into a network, and subsequently, either paralyzes the network functionalities for unauthorized usages or maintain its functionalities in the presence of a verified passport. Such a desired network behavior is successfully demonstrated in a number of implementation schemes, which provide reliable, preventive and timely protections against tens of thousands of fake-passport deceptions. Extensive experiments also show that the deep neural…
In the field of information technology, information security technologies hold a special place. They ensure the security of the use of information technology. One of the urgent tasks is the protection of electronic documents during their transfer in information systems. This paper proposes a technology for protecting electronic documents containing digital images. The main idea is that the electronic document authorship protection can be implemented by digital watermark embedding in the images that are contained in this document. The paper considers three cases of using the proposed technology…
The security of patient information is important during the transfer of medical data. A hybrid spatial domain watermarking algorithm that includes encryption, integrity protection and steganography is proposed to strengthen the information originality based on the authentication. The proposed algorithm checks whether the information of patients has been deliberately changed or not. The created code is distributed at every pixel of the medical image and not only in the regions of noninterest pixels, while the image details are still preserved. To enhance the security of the watermarking code, "…
THEMIS: Towards Practical Intellectual Property Protection for Post-Deployment On-Device Deep Learning Models
The University of Melbourne（墨尔本大学）；The University of Western Australia（西澳大学）；City University of Hong Kong（香港城市大学）；Technical University of Munich（慕尼黑工业大学）
Creating a state-of-the-art deep-learning system requires vast amounts of data, expertise, and hardware, yet research into embedding copyright protection for neural networks has been limited. One of the main methods for achieving such protection involves relying on the susceptibility of neural networks to backdoor attacks, but the robustness of these tactics has been primarily evaluated against pruning, fine-tuning, and model inversion attacks. In this work, we propose a neural network "laundering" algorithm to remove black-box backdoor watermarks from neural networks even when the adversary h…
Due to the recent demand of 3-D models in several applications like medical imaging, video games, among others, the necessity of implementing 3-D mesh watermarking schemes aiming to protect copyright has increased considerably. The majority of robust 3-D watermarking techniques have essentially focused on the robustness against attacks while the imperceptibility of these techniques is still a real issue. In this context, a blind robust 3-D mesh watermarking method based on mesh saliency and Quantization Index Modulation (QIM) for Copyright protection is proposed. The watermark is embedded by q…
The rapid development of multimedia and internet allows for wide distribution of digital media data. It becomes much easier to edit, modify and duplicate digital information besides that, digital documents are also easy to copy and distribute, therefore it will be faced by many threats. It is a big security and privacy issue. Another problem with digital document and video is that undetectable modifications can be made with very simple and widely available equipment, which put the digital material for evidential purposes under question With the large flood of information and the development of…
Due to the recent demand of 3-D meshes in a wide range of applications such as video games, medical imaging, film special effect making, computer-aided design (CAD), among others, the necessity of implementing 3-D mesh watermarking schemes aiming to protect copyright has increased in the last decade. Nowadays, the majority of robust 3-D watermarking approaches have mainly focused on the robustness against attacks while the imperceptibility of these techniques is still a serious challenge. In this context, a blind robust 3-D mesh watermarking method based on mesh saliency and scalar Costa schem…
Software watermarking involves embedding a unique identifier or, equivalently, a watermark value within a software to prove owner's authenticity and thus to prevent or discourage copyright infringement. Towards the embedding process, several graph theoretic watermarking algorithmic techniques encode the watermark values as graph structures and embed them in application programs. Recently, we presented an efficient codec system for encoding a watermark number $w$ as a reducible permutation graph $F[π^*]$ through the use of self-inverting permutations $π^*$. In this paper, we propose a dynamic w…
