难度校准就是我们所需的一切吗?迈向更实用的成员推理攻击
Is Difficulty Calibration All We Need? Towards More Practical Membership Inference Attacks
AI总结:
本文揭示难度校准在成员推理攻击中的固有缺陷,提出查询与计算高效的RAPID攻击方法,通过直接重用原始成员分数来修正校准错误,在9个数据集和5种模型架构上超越LiRA等现有最优攻击。
摘要:
近年来,机器学习模型对成员推理攻击(Membership Inference Attacks, MIAs)的脆弱性引起了广泛关注。这些攻击旨在判断某个数据样本是否属于模型的训练集。近期研究主要聚焦于基于参考的攻击方法,这类方法利用独立训练的参考模型进行难度校准。尽管实证研究表明其有效性,但我们对它在何种情况下成功或失败的理解仍存在显著不足。在本文中,我们朝着更深入理解难度校准的作用迈出了进一步的一步。我们的观察揭示了校准方法固有的局限性,这些局限性导致非成员样本被错误分类,并且在高损失样本上性能表现欠佳。我们进一步发现,这些错误源于对潜在分布的不完善采样,以及成员分数对模型参数的强依赖性。通过阐明这些问题,我们提出了RAPID:一种查询高效且计算高效的成员推理攻击方法,它直接重新利用原始成员分数来缓解难度校准中的错误。我们的实验结果涵盖9个数据集和5种模型架构,表明RAPID在不同评估指标上均优于先前最先进的攻击方法(如LiRA和Canary offline),同时保持计算高效。我们的观察与分析挑战了当前高精度推理中难度校准的事实范式,促使人们更加关注成员推理攻击在更实际场景中所带来的持续风险。
英文摘要:
The vulnerability of machine learning models to Membership Inference Attacks (MIAs) has garnered considerable attention in recent years. These attacks determine whether a data sample belongs to the model's training set or not. Recent research has focused on reference-based attacks, which leverage difficulty calibration with independently trained reference models. While empirical studies have demonstrated its effectiveness, there is a notable gap in our understanding of the circumstances under which it succeeds or fails. In this paper, we take a further step towards a deeper understanding of the role of difficulty calibration. Our observations reveal inherent limitations in calibration methods, leading to the misclassification of non-members and suboptimal performance, particularly on high-loss samples. We further identify that these errors stem from an imperfect sampling of the potential distribution and a strong dependence of membership scores on the model parameters. By shedding light on these issues, we propose RAPID: a query-efficient and computation-efficient MIA that directly \textbf{R}e-lever\textbf{A}ges the original membershi\textbf{P} scores to m\textbf{I}tigate the errors in \textbf{D}ifficulty calibration. Our experimental results, spanning 9 datasets and 5 model architectures, demonstrate that RAPID outperforms previous state-of-the-art attacks (e.g., LiRA and Canary offline) across different metrics while remaining computationally efficient. Our observations and analysis challenge the current de facto paradigm of difficulty calibration in high-precision inference, encouraging greater attention to the persistent risks posed by MIAs in more practical scenarios.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。