一个对象，多重谎言：面向统一视觉语言模型的跨任务对抗攻击基准
One Object, Multiple Lies: A Benchmark for Cross-task Adversarial Attack on Unified Vision-Language Models
AI总结：
本文针对统一视觉语言模型的安全挑战，提出了CrossVLAD基准数据集和CRAFT攻击方法，通过在多任务中一致操纵目标对象分类，实现了高效的跨任务对抗攻击。
摘要：
统一视觉语言模型（VLMs）近期展现出显著进展，使单一模型能够通过共享计算架构内的不同指令灵活处理多种任务。这种基于指令的控制机制带来了独特的安全挑战，因为对抗性输入必须在可能被不可预测地应用于处理同一恶意内容的多种任务指令中保持有效。本文引入了CrossVLAD，这是一个从MSCOCO中精心挑选并借助GPT-4辅助进行标注的新基准数据集，用于系统评估统一VLMs上的跨任务对抗攻击。CrossVLAD以目标对象改变为核心目标——即在四个下游任务中一致地操纵目标对象的分类——并提出了一种新颖的成功率指标，用于衡量所有任务中同时发生错误分类的情况，为对抗性可迁移性提供了严格评估。为应对这一挑战，我们提出了CRAFT（Cross-task Region-based Attack Framework with Token-alignment），一种高效的以区域为中心的攻击方法。在Florence-2和其他流行的统一VLMs上的大量实验表明，我们的方法在整体跨任务攻击性能和有针对性的目标对象改变成功率方面均优于现有方法，凸显了其在跨多种任务对统一VLMs进行对抗性影响的有效性。
英文摘要：
Unified vision-language models(VLMs) have recently shown remarkable progress, enabling a single model to flexibly address diverse tasks through different instructions within a shared computational architecture. This instruction-based control mechanism creates unique security challenges, as adversarial inputs must remain effective across multiple task instructions that may be unpredictably applied to process the same malicious content. In this paper, we introduce CrossVLAD, a new benchmark dataset carefully curated from MSCOCO with GPT-4-assisted annotations for systematically evaluating cross-task adversarial attacks on unified VLMs. CrossVLAD centers on the object-change objective-consistently manipulating a target object's classification across four downstream tasks-and proposes a novel success rate metric that measures simultaneous misclassification across all tasks, providing a rigorous evaluation of adversarial transferability. To tackle this challenge, we present CRAFT (Cross-task Region-based Attack Framework with Token-alignment), an efficient region-centric attack method. Extensive experiments on Florence-2 and other popular unified VLMs demonstrate that our method outperforms existing approaches in both overall cross-task attack performance and targeted object-change success rates, highlighting its effectiveness in adversarially influencing unified VLMs across diverse tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
