发表机构
MBZUAI; Michigan State University（穆罕默德·本·扎耶德人工智能大学; 密歇根州立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对个性化视觉-语言模型中的身份绑定与识别隐私风险，提出基于身份原型扰动破坏的图像级防御方法Anti-Persona，在保持视觉保真度下实现高达95%的保护率。
AI 中文摘要
少样本个性化使大型视觉-语言模型（LVLMs）能够学习用户特定的视觉概念，用于个性化检索和主体感知查询等应用。然而，这也带来了隐私风险：攻击者可以从少量参考图像中绑定目标身份，随后通过自然语言查询在新图像中检测该身份。我们提出了Anti-Persona，一种针对个性化LVLMs中未授权身份绑定与识别的图像级防御方法。我们的关键洞察在于，身份个性化依赖于多个参考图像共享的视觉特征。我们将这些特征聚合为身份原型，并优化视觉上微妙的扰动，以破坏视觉编码器空间中的原型对齐。空间平滑和低频保留进一步提升了视觉保真度以及对图像压缩的实际鲁棒性。所得到的保护不依赖于特定提示，并同时支持主动反个性化和被动图像保护。在两个代表性个性化LVLMs上的实验表明，保护率高达95.0%，同时保持了视觉保真度。该方法在提示变化和所评估的身份查询任务中保持稳定，并在编码器不匹配下改善了黑盒迁移。
英文摘要
Few-shot personalization enables large vision--language models (LVLMs) to learn user-specific visual concepts for applications such as personalized retrieval and subject-aware querying. However, it also creates a privacy risk: an adversary can bind a target identity from a few reference images and subsequently detect that identity in new images through natural-language queries. We introduce Anti-Persona, an image-level defense against unauthorized identity binding and recognition in personalized LVLMs. Our key insight is that identity personalization relies on visual features shared across multiple reference images. We aggregate these features into an identity prototype and optimize visually subtle perturbations that disrupt prototype alignment in the vision-encoder space. Spatial smoothing and low-frequency preservation further promote visual fidelity and practical resilience to image compression. The resulting protection does not depend on a specific prompt and supports both proactive anti-personalization and reactive image protection. Experiments on two representative personalized LVLMs demonstrate protection rates of up to $95.0\%$ while preserving visual fidelity. The method remains stable across prompt variations and evaluated identity-query tasks, and improves black-box transfer under encoder mismatch.
CommentsCode available at https://github.com/iabh1shekbasu/anti-persona
