arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.04339cs.LGcs.AIcs.CR
收藏

一框架适用于所有:生成模型的跨模态成员推理

One Framework for All: Cross-Modal Membership Inference for Generative Models

Dayong Ye, Tainqing Zhu, Kun Gao, Junhao Liu, Yichuan Chen, Shuai Zhou, Hengzhu Liu, Bo Liu, Wanlei Zhou

AI总结:

研究生成模型的跨模态成员推理问题,基于生成模型输出分布可近似训练数据分布的特性,在共享嵌入空间建模,通过似然比测试推理,贡献统一框架,性能优于现有方法。

摘要:

跨文本到文本、文本到图像和图像到文本模态的大型生成模型存在重大隐私风险,成员推理攻击是一个基本威胁。现有方法孤立处理这三类生成模型且不交叉适用。本文提出首个统一成员推理框架,基于关键观察建模,通过似然比测试进行推理,实验表明性能优于现有方法。

英文摘要:

Large generative models across text-to-text, text-to-image, and image-to-text modalities have been shown to pose significant privacy risks. One fundamental threat is membership inference attacks (MIA), which aim to determine whether a given data point was used in a model's training set. Although prior work has investigated MIAs against these three classes of generative models, existing approaches treat them in isolation and are not cross-applicable, thereby limiting their real-world utility. To address this limitation, we present the first comprehensive study of a unified membership inference framework that applies across text-to-text, text-to-image, and image-to-text modalities. Our approach is grounded in a key modality-agnostic observation: the output distribution of a generative model can approximate its training data distribution. Leveraging this property, we model the distributions of model-generated outputs and auxiliary non-member samples in a shared embedding space, and perform membership inference via likelihood ratio testing. We conduct extensive experiments in a strict black-box setting under both partial-knowledge and zero-knowledge threat models, and evaluate membership inference against both fine-tuning and pre-training data. Experimental results demonstrate our approach's superior performance in comparison to existing state-of-the-art methods, which are typically optimized for a single model class.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑