arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.29454cs.LG
收藏

用于评估机器学习中成员推断攻击的全流程框架

A Full-Pipeline Framework for Evaluating Membership Inference Attacks in Machine Learning

City University of Hong Kong(香港城市大学);Shanghai Jiao Tong University(上海交通大学)

Ding Chen, Xinwen Cheng, Xuyang Zhong, Xinping Chen, Xiaolin Huang, Chen Liu

AI总结:

提出一个涵盖数据、架构、算法和后训练模块的全流程评估框架,系统分析不同上下文对成员推断攻击效果的影响,并通过标准化威胁模型和互补指标提供实用指南。

摘要:

虽然成员推断攻击(MIAs)是识别训练数据的主流方法,但其应用已扩展到隐私审计和机器遗忘。然而,该领域缺乏一个系统性的框架来评估不同上下文如何影响MIA的效果。没有这样的特征描述,实践者可能会部署在基准测试中表现良好但在面对特定真实世界数据集的细微差别时变得统计上无关的算法。为了弥合这一差距并提供可操作的见解,我们引入了一个全面的评估框架,该框架系统地描述了整个机器学习流程(包括数据、架构、算法和后训练模块)中的隐私风险。我们的框架旨在固有地捕捉多样化的操作上下文,严格评估了在广泛训练配置下的最先进MIA。为了考虑真实世界部署中不同的误分类成本,我们采用了三个互补指标:对称成本下的平衡准确率,以及低FPR下的TPR(或低FNR下的TNR)用于严格惩罚误报或漏检的非对称场景。此外,认识到现有MIA假设不同的对手能力,我们形式化了两种标准化的威胁模型,并将这些攻击调整为相应的变体,以确保公平的基准测试。大量的实证评估表明,特定MIA方法的效果高度依赖于假设的威胁模型和选择的评估指标。最终,我们将这些发现提炼为可操作的指南,并提供一个即用的审计工具包,使实践者能够进行更好的隐私评估。

英文摘要:

While Membership Inference Attacks (MIAs) are the prevailing method for identifying training data, their application has expanded into privacy auditing and machine unlearning. Nevertheless, the field lacks a systematic framework for evaluating how different contexts affect MIA efficacy. Without such a characterization, practitioners risk deploying algorithms that perform well on benchmarks but become statistically irrelevant when faced with the nuances of specific, real-world datasets. To bridge this gap and provide actionable insights, we introduce a comprehensive evaluation framework that systematically characterizes privacy risks across the entire machine learning pipeline, spanning data, architectures, algorithms, and post-training modules. Designed to inherently capture diverse operational contexts, our framework rigorously evaluates state-of-the-art MIAs across a broad spectrum of training configurations. To account for varying misclassification costs in real-world deployments, we employ three complementary metrics: Balanced Accuracy for symmetric costs, alongside TPR at low FPR (or TNR at low FNR) for asymmetric scenarios where false alarms or missed detections are strictly penalized. Furthermore, recognizing that existing MIAs assume divergent adversary capabilities, we formalize two standardized threat models and adapt these attacks into corresponding variants to ensure an equitable benchmark. Extensive empirical evaluations demonstrate that the efficacy of specific MIA methodologies is highly sensitive to the assumed threat models and chosen evaluation metrics. Ultimately, we distill these findings into actionable guidelines and provide a ready-to-use auditing toolkit, empowering practitioners to conduct better privacy assessments.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑