发表机构
School of Cyber Science and Engineering, Huazhong University of Science and Technology; College of Computer Science, Chongqing University; School of Software and engineering, Huazhong University of Science and Technology; School of Information and Communication Technology, Griffith University（华中科技大学网络空间安全学院; 重庆大学计算机科学学院; 华中科技大学软件工程学院; 格里菲斯大学信息与通信技术学院）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
研究针对视觉语言模型基于文本的隐私攻击问题，提出CloakDiff框架，结合扩散对抗编辑与可逆网络生成不可察觉的对抗性示例，设计EDM启发式采样提高保真度，实验证明该框架能实现多模态隐私保护且具高视觉质量和可逆性。
AI 中文摘要
视觉语言模型（VLMs）提供强大的多模态能力，但也使用户面临基于文本的隐私攻击，对手通过抓取在线照片并查询VLMs来提取敏感属性。现有的可逆对抗性示例（RAE）方法在纯视觉任务中保护图像，但在多模态设置中失败，当前针对VLMs的对抗性示例依赖于严重降低视觉质量的高频噪声。我们提出了CloakDiff，这是第一个针对VLMs中基于文本的查询攻击进行可逆、高保真隐私保护的框架。CloakDiff通过将基于扩散的对抗性编辑与用于无损恢复的嵌入原始图像的可逆网络相结合来生成不可察觉的对抗性示例。它扰动像素空间嵌入并操纵潜在的交叉注意力图，以确保在保留全局视觉结构的同时具有强大的跨模型和跨提示可转移性。为了进一步提高保真度，我们设计了EDM启发式采样，这是一种用于对抗性指导的有原则的扩散调度。在多个数据集和VLMs上的实验表明，CloakDiff提供了具有高视觉质量和可逆性的多模态隐私保护。
英文摘要
Vision Language Models (VLMs) offer powerful multimodal ability but also expose users to text-based privacy attacks where adversaries crawl online photos and query VLMs to extract sensitive attributes. Existing reversible adversarial example (RAE) methods protect images in purely visual tasks but fail in multimodal settings, and current adversarial examples on VLMs rely on high frequency noise that severely degrades visual quality. We propose CloakDiff, the first framework for reversible, high fidelity privacy protection against text-based query attacks in VLMs. CloakDiff produces imperceptible adversarial examples by combining diffusion based adversarial editing with an invertible network that embeds the original image for lossless recovery. It perturbs both pixel space embeddings and manipulates latent cross attention maps to ensure strong cross-model and cross-prompt transferability while preserving global visual structure. To further enhance fidelity, we design EDM Heuristic Sampling, a principled diffusion schedule for adversarial guidance. Experiments on multiple datasets and VLMs demonstrate that CloakDiff delivers multimodal privacy preservation with high visual quality and reversibility.
CommentsAccepted by ACM MM 2026
