arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.02747cs.LGcs.CR
收藏

曲率线索:利用输入损失曲率解码深度学习隐私

Curvature Clues: Decoding Deep Learning Privacy with Input Loss Curvature

Purdue University(普渡大学)

Deepak Ravikumar, Efstathia Soufleri, Kaushik Roy

AI总结:

本文研究输入损失曲率在训练集与测试集间的差异,提出基于曲率的黑盒成员推理攻击,并在多个数据集上验证其优于现有方法。

摘要:

本文探讨深度神经网络中损失关于输入数据的曲率性质。损失关于输入的曲率(称为输入损失曲率)是损失关于输入的 Hessian 矩阵的迹。我们研究输入损失曲率在训练集和测试集之间的变化,及其对训练-测试可区分性的影响。我们构建了一个理论框架,基于隐私和训练集大小推导出训练-测试可区分性的上界。这一新见解推动了利用输入损失曲率的新型黑盒成员推理攻击的发展。我们通过在计算机视觉分类任务中的实验验证了理论发现,证明输入损失曲率在成员推理有效性上超越了现有方法。我们的分析揭示了成员推理攻击(MIA)方法的性能如何随训练集大小变化,表明基于曲率的 MIA 在足够大的数据集上优于其他方法。这一条件通常被真实数据集满足,正如我们在 CIFAR10、CIFAR100 和 ImageNet 上的结果所示。这些发现不仅增进了我们对深度神经网络行为的理解,还提升了测试机器学习中隐私保护技术的能力。

英文摘要:

In this paper, we explore the properties of loss curvature with respect to input data in deep neural networks. Curvature of loss with respect to input (termed input loss curvature) is the trace of the Hessian of the loss with respect to the input. We investigate how input loss curvature varies between train and test sets, and its implications for train-test distinguishability. We develop a theoretical framework that derives an upper bound on the train-test distinguishability based on privacy and the size of the training set. This novel insight fuels the development of a new black box membership inference attack utilizing input loss curvature. We validate our theoretical findings through experiments in computer vision classification tasks, demonstrating that input loss curvature surpasses existing methods in membership inference effectiveness. Our analysis highlights how the performance of membership inference attack (MIA) methods varies with the size of the training set, showing that curvature-based MIA outperforms other methods on sufficiently large datasets. This condition is often met by real datasets, as demonstrated by our results on CIFAR10, CIFAR100, and ImageNet. These findings not only advance our understanding of deep neural network behavior but also improve the ability to test privacy-preserving techniques in machine learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑