ETA：在推理时先评估后对齐视觉语言模型的安全性
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time
AI总结：
本文提出ETA框架，在推理时先评估视觉输入与输出以建立安全意识，再通过干扰前缀和句子级best-of-N对齐不安全行为，显著降低不安全率并提升有用性。
摘要：
视觉语言模型（VLMs）已成为多模态智能的重要骨干，但显著的安全挑战限制了它们的实际应用。尽管文本输入通常能得到有效防护，对抗性视觉输入却可以轻易绕过VLM的防御机制。现有防御方法要么资源密集，需要大量数据和计算，要么无法在响应中同时保证安全性和有用性。为解决这些局限，我们提出了一种新颖的两阶段推理时对齐框架，即先评估后对齐（ETA）：1）评估输入视觉内容和输出响应，以在多模态场景中建立稳健的安全意识；2）通过以干扰前缀条件化VLMs的生成分布，并执行句子级best-of-N来搜索最无害且最有帮助的生成路径，从而在浅层和深层两个层面同时对不安全行为进行对齐。大量实验表明，ETA在无害性、有用性和效率方面均优于基线方法，在跨模态攻击中将不安全率降低了87.5%，并在GPT-4有用性评估中取得了96.6%的胜平率。代码已公开于https://github.com/DripNowhy/ETA。
英文摘要：
Vision Language Models (VLMs) have become essential backbones for multimodal intelligence, yet significant safety challenges limit their real-world application. While textual inputs are often effectively safeguarded, adversarial visual inputs can easily bypass VLM defense mechanisms. Existing defense methods are either resource-intensive, requiring substantial data and compute, or fail to simultaneously ensure safety and usefulness in responses. To address these limitations, we propose a novel two-phase inference-time alignment framework, Evaluating Then Aligning (ETA): 1) Evaluating input visual contents and output responses to establish a robust safety awareness in multimodal settings, and 2) Aligning unsafe behaviors at both shallow and deep levels by conditioning the VLMs' generative distribution with an interference prefix and performing sentence-level best-of-N to search the most harmless and helpful generation paths. Extensive experiments show that ETA outperforms baseline methods in terms of harmlessness, helpfulness, and efficiency, reducing the unsafe rate by 87.5% in cross-modality attacks and achieving 96.6% win-ties in GPT-4 helpfulness evaluation. The code is publicly available at https://github.com/DripNowhy/ETA.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
