arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.03392cs.LGcs.AI
收藏

面向大语言模型成员推断攻击的自动校准方法

Automatic Calibration for Membership Inference Attack on Large Language Models

Department of Computer Science, Wayne State University(韦恩州立大学计算机科学系);Department of Computer Science, Oakland University(奥克兰大学计算机科学系)

Saleh Zare Zade, Yao Qiang, Xiangyu Zhou, Hui Zhu, Mohammad Amin Roshani, Prashant Khanduri, Dongxiao Zhu

AI总结:

本文提出自动校准成员推断攻击框架ACMIA,通过可调温度校准输出概率并扩大成员与非成员概率差距,在多种模型访问级别下提升成员推断的可靠性,并在多个基准上超越现有方法。

摘要:

成员推断攻击(Membership Inference Attacks, MIAs)近来被用于判断特定文本是否属于大语言模型(Large Language Models, LLMs)的预训练数据。然而,现有方法常常将非成员误判为成员,导致较高的假阳性率,或者依赖额外的参考模型进行概率校准,从而限制了其实用性。为克服这些挑战,我们提出了一种新颖的框架,称为自动校准成员推断攻击(Automatic Calibration Membership Inference Attack, ACMIA),该框架利用可调温度来有效校准输出概率。这一方法受到我们对大语言模型预训练过程中最大似然估计的理论洞见所启发。我们以三种配置引入 ACMIA,旨在适应不同级别的模型访问权限,并扩大成员与非成员之间的概率差距,从而提高成员推断的可靠性和鲁棒性。在多个开源大语言模型上的大量实验表明,我们提出的攻击方法具有高效性、鲁棒性和泛化能力,在三个广泛使用的基准上超越了当前最先进的基线方法。我们的代码可在以下地址获取:\href{https://github.com/Salehzz/ACMIA}{\textcolor{blue}{Github}}。

英文摘要:

Membership Inference Attacks (MIAs) have recently been employed to determine whether a specific text was part of the pre-training data of Large Language Models (LLMs). However, existing methods often misinfer non-members as members, leading to a high false positive rate, or depend on additional reference models for probability calibration, which limits their practicality. To overcome these challenges, we introduce a novel framework called Automatic Calibration Membership Inference Attack (ACMIA), which utilizes a tunable temperature to calibrate output probabilities effectively. This approach is inspired by our theoretical insights into maximum likelihood estimation during the pre-training of LLMs. We introduce ACMIA in three configurations designed to accommodate different levels of model access and increase the probability gap between members and non-members, improving the reliability and robustness of membership inference. Extensive experiments on various open-source LLMs demonstrate that our proposed attack is highly effective, robust, and generalizable, surpassing state-of-the-art baselines across three widely used benchmarks. Our code is available at: \href{https://github.com/Salehzz/ACMIA}{\textcolor{blue}{Github}}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑