通过率约束变分自编码器净化不可学习样本
Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders
AI总结:
本文提出解耦变分自编码器D-VAE,利用率约束VAE抑制不可学习样本中的扰动,并通过两阶段净化方法实现高效训练前净化,在多个数据集上验证了有效性。
摘要:
不可学习样本(UEs)通过对正确标注的训练样本进行细微修改,试图最大化测试误差。针对这些投毒攻击的防御方法可根据是否在训练过程中采用特定干预措施进行分类。第一种方法是训练时防御,例如对抗训练,它可以减轻投毒影响,但计算开销很大。另一种方法是训练前净化,例如图像短压缩,它由若干简单压缩操作组成,但在处理各种不可学习样本时常常面临挑战。我们的工作提供了一种新颖的解耦机制,以构建高效的训练前净化方法。首先,我们揭示了率约束变分自编码器(VAEs),证明其具有明显的抑制不可学习样本中扰动的倾向。随后,我们对该现象进行了理论分析。基于这些见解,我们引入了一种解耦变分自编码器(D-VAE),它能够利用可学习的类别级嵌入来解耦扰动。在此网络基础上,自然发展出了一种两阶段净化方法。第一阶段侧重于粗略消除扰动,而第二阶段生成精细的、无毒的结果,确保在各种场景下的有效性和鲁棒性。大量实验证明了我们的方法在CIFAR-10、CIFAR-100以及一个包含100类的ImageNet子集上的卓越性能。代码可在https://github.com/yuyi-sd/D-VAE获取。
英文摘要:
Unlearnable examples (UEs) seek to maximize testing error by making subtle modifications to training examples that are correctly labeled. Defenses against these poisoning attacks can be categorized based on whether specific interventions are adopted during training. The first approach is training-time defense, such as adversarial training, which can mitigate poisoning effects but is computationally intensive. The other approach is pre-training purification, e.g., image short squeezing, which consists of several simple compressions but often encounters challenges in dealing with various UEs. Our work provides a novel disentanglement mechanism to build an efficient pre-training purification method. Firstly, we uncover rate-constrained variational autoencoders (VAEs), demonstrating a clear tendency to suppress the perturbations in UEs. We subsequently conduct a theoretical analysis for this phenomenon. Building upon these insights, we introduce a disentangle variational autoencoder (D-VAE), capable of disentangling the perturbations with learnable class-wise embeddings. Based on this network, a two-stage purification approach is naturally developed. The first stage focuses on roughly eliminating perturbations, while the second stage produces refined, poison-free results, ensuring effectiveness and robustness across various scenarios. Extensive experiments demonstrate the remarkable performance of our method across CIFAR-10, CIFAR-100, and a 100-class ImageNet-subset. Code is available at https://github.com/yuyi-sd/D-VAE.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。