arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2203.11894cs.CVcs.AIcs.CRcs.DCcs.LG
收藏

GradViT:视觉Transformer的梯度反演

GradViT: Gradient Inversion of Vision Transformers

NVIDIA(英伟达)

Ali Hatamizadeh, Hongxu Yin, Holger Roth, Wenqi Li, Jan Kautz, Daguang Xu, Pavlo Molchanov

摘要:

在这项工作中,我们证明了视觉Transformer(ViTs)的脆弱性,基于梯度的反转攻击。在此攻击期间,给定模型权重和相应的梯度,重建原始数据批。我们介绍了一种名为GradViT的方法,该方法通过迭代过程将随机噪声优化为自然的图像。优化目标包括(i)梯度匹配的损失,(ii)以预训练的CNN模型的批量归一化统计数据的距离形式的图像先验,以及(iii)补丁上的总变差正则化以指导正确的恢复位置。我们提出了一个独特的损失调度函数,以克服局部极小值在优化过程中。我们在ImageNet 1 K和MS-Celeb-1 M数据集上评估了GadViT,并观察到前所未有的高保真度和与原始(隐藏)数据的接近度。在分析过程中,我们发现,由于注意力机制的存在,视觉Transformer比以前研究的CNN更容易受到攻击。我们的方法在定性和定量指标上展示了梯度反演的最新结果。项目页面https://gradvit.github.io/。

英文摘要:

In this work we demonstrate the vulnerability of vision transformers (ViTs) to gradient-based inversion attacks. During this attack, the original data batch is reconstructed given model weights and the corresponding gradients. We introduce a method, named GradViT, that optimizes random noise into naturally looking images via an iterative process. The optimization objective consists of (i) a loss on matching the gradients, (ii) image prior in the form of distance to batch-normalization statistics of a pretrained CNN model, and (iii) a total variation regularization on patches to guide correct recovery locations. We propose a unique loss scheduling function to overcome local minima during optimization. We evaluate GadViT on ImageNet1K and MS-Celeb-1M datasets, and observe unprecedentedly high fidelity and closeness to the original (hidden) data. During the analysis we find that vision transformers are significantly more vulnerable than previously studied CNNs due to the presence of the attention mechanism. Our method demonstrates new state-of-the-art results for gradient inversion in both qualitative and quantitative metrics. Project page at https://gradvit.github.io/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑