Convolutional Neural Networks (CNNs) and their quantized counterparts are vulnerable to extraction attacks, posing a significant threat of IP theft. Yet, the robustness of quantized models against these attacks is little studied compared to large models. Previous defenses propose to inject calculated noise into the prediction probabilities. However, these defenses are limited since they are not incorporated during the model design and are only added as an afterthought after training. Additionally, most defense techniques are computationally expensive and often have unrealistic assumptions about the victim model that are not feasible in edge device implementations and do not apply to quantized models. In this paper, we propose DivQAT, a novel algorithm to train quantized CNNs based on Quantization Aware Training (QAT) aiming to enhance their robustness against extraction attacks. To the best of our knowledge, our technique is the first to modify the quantization process to integrate a model extraction defense into the training process. Through empirical validation on benchmark vision datasets, we demonstrate the efficacy of our technique in defending against model extraction attacks without compromising model accuracy. Furthermore, combining our quantization technique with other defense mechanisms improves their effectiveness compared to traditional QAT.
Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correction
Hubei Key Laboratory of Trasportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology（湖北交通物联网重点实验室，计算机科学与人工智能学院，武汉理工大学）；School of Computer Science, Wuhan University（计算机科学学院，武汉大学）
Audio codecs based on discretized neural autoencoders have recently been developed and shown to provide significantly higher compression levels for comparable quality speech output. However, these models are tightly coupled with speech content, and produce unintended outputs in noisy conditions. Based on VQ-VAE autoencoders with WaveRNN decoders, we develop compressor-enhancer encoders and accompanying decoders, and show that they operate well in noisy conditions. We also observe that a compressor-enhancer model performs better on clean speech inputs than a compressor model trained only on cle…
Deep quantization of neural networks (below eight bits) offers significant promise in reducing their compute and storage cost. Albeit alluring, without special techniques for training and optimization, deep quantization results in significant accuracy loss. To further mitigate this loss, we propose a novel sinusoidal regularization, called SinReQ1, for deep quantized training. SinReQ adds a periodic term to the original objective function of the underlying training algorithm. SinReQ exploits the periodicity, differentiability, and the desired convexity profile in sinusoidal functions to automa…
University of California San Diego（加州大学圣地亚哥分校）；CNRS（法国国家科学研究中心）；Université de Bordeaux（波尔多大学）；Bordeaux INP（波尔多国立理工学院）
As deep neural networks make their ways into different domains, their compute efficiency is becoming a first-order constraint. Deep quantization, which reduces the bitwidth of the operations (below 8 bits), offers a unique opportunity as it can reduce both the storage and compute requirements of the network super-linearly. However, if not employed with diligence, this can lead to significant accuracy loss. Due to the strong inter-dependence between layers and exhibiting different characteristics across the same network, choosing an optimal bitwidth per layer granularity is not a straight forwa…
A Runtime-Adaptive Transformer Neural Network Accelerator on FPGAs
Department of Electrical Engineering and Computer Science, University of Arkansas（电气工程与计算机科学系，亚拉巴马大学）；Department of Computer Science and Engineering, University of South Carolina（计算机科学与工程系，南卡罗来纳大学）
Saliency Assisted Quantization for Neural Networks
Amirkabir University of Technology（阿米尔卡比尔理工大学）；University of St. Thomas, Minnesota（明尼苏达州圣托马斯大学）；Isfahan University of Technology（伊斯法罕理工大学）；University of Cincinnati（辛辛那提大学）；University of Illinois Chicago（芝加哥伊利诺伊大学）
