PII-VisBench: 在可见性连续体上评估视觉语言模型中个人可识别信息安全性
PII-VisBench: Evaluating Personally Identifiable Information Safety in Vision Language Models Along a Continuum of Visibility
AI总结：
PII-VisBench 评估视觉语言模型在可见性连续体上对个人可识别信息安全性的保护，通过4000个探测器分析不同可见性水平下的隐私泄露情况。
摘要：
视觉语言模型（VLMs）正在越来越多地集成到隐私关键领域中，但现有的个人可识别信息（PII）泄漏评估大多将隐私视为静态提取任务，并忽略了个体在线存在——他们在线上可用数据量——如何影响隐私对齐。我们介绍了PII-VisBench，一个包含4000个独特探测器的新基准，旨在通过在线存在连续体评估VLM的安全性。该基准将200个受试者分为四个可见性类别：高、中、低和零——基于其在线信息的范围和性质。我们基于两个关键指标评估了18个开源VLMs（0.3B-32B）：被拒绝的PII探测查询百分比（拒绝率）和非拒绝响应中被标记为包含PII的比例（条件PII披露率）。在模型之间，我们观察到一致的模式：随着受试者可见性的下降，拒绝率增加，PII披露率下降（9.10%高到5.34%低）。我们发现，模型更倾向于为高可见性受试者披露PII，同时存在显著的模型家族异质性和PII类型差异。最后，改写和 Jailbreak 风格的提示暴露了攻击和模型依赖的失败，促使可见性意识的安全评估和训练干预。
英文摘要：
Vision Language Models (VLMs) are increasingly integrated into privacy-critical domains, yet existing evaluations of personally identifiable information (PII) leakage largely treat privacy as a static extraction task and ignore how a subject's online presence--the volume of their data available online--influences privacy alignment. We introduce PII-VisBench, a novel benchmark containing 4000 unique probes designed to evaluate VLM safety through the continuum of online presence. The benchmark stratifies 200 subjects into four visibility categories: high, medium, low, and zero--based on the extent and nature of their information available online. We evaluate 18 open-source VLMs (0.3B-32B) based on two key metrics: percentage of PII probing queries refused (Refusal Rate) and the fraction of non-refusal responses flagged for containing PII (Conditional PII Disclosure Rate). Across models, we observe a consistent pattern: refusals increase and PII disclosures decrease (9.10% high to 5.34% low) as subject visibility drops. We identify that models are more likely to disclose PII for high-visibility subjects, alongside substantial model-family heterogeneity and PII-type disparities. Finally, paraphrasing and jailbreak-style prompts expose attack and model-dependent failures, motivating visibility-aware safety evaluation and training interventions.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
