面向上下文感知安全的多模态大语言模型解码引导
Steering Multimodal Large Language Models Decoding for Context-Aware Safety
AI总结：
针对多模态大语言模型安全对齐的平衡难题，提出SafeCoDe解码框架，通过两阶段机制优化上下文敏感的弃权行为，在多基准实验中有效提升安全性能且保留模型有用性。
摘要：
多模态大语言模型（Multimodal Large Language Models, MLLMs）正越来越多地部署在实际应用中，但其做出上下文感知安全决策的能力仍有限。现有方法往往无法平衡过度敏感（无理由拒绝良性查询）与欠敏感（未检测到基于视觉的风险），在安全对齐方面存在持续差距。为解决该问题，我们提出安全感知对比解码框架（Safety-aware Contrastive Decoding, SafeCoDe），这是一种轻量且与模型无关的解码框架，可基于多模态上下文动态调整token生成。SafeCoDe分两阶段运行：（1）对比解码机制，通过对比真实图像与高斯噪声图像，突出对视觉上下文敏感的token；（2）全局感知token调制策略，将场景级推理与token级调整相结合，根据预测的安全判定结果调整弃权（不执行）行为。在涵盖欠敏感、过度敏感及通用安全评估的各类MLLM架构与安全基准上开展的大量实验表明，SafeCoDe可在保持模型有用性的同时，持续优化上下文敏感的弃权（不执行）行为。
英文摘要：
Multimodal Large Language Models (MLLMs) are increasingly deployed in real-world applications, yet their ability to make context-aware safety decisions remains limited. Existing methods often fail to balance oversensitivity (unjustified refusals of benign queries) and undersensitivity (missed detection of visually grounded risks), leaving a persistent gap in safety alignment. To address this issue, we introduce Safety-aware Contrastive Decoding (SafeCoDe), a lightweight and model-agnostic decoding framework that dynamically adjusts token generation based on multimodal context. SafeCoDe operates in two stages: (1) a contrastive decoding mechanism that highlights tokens sensitive to visual context by contrasting real and Gaussian-noised images, and (2) a global-aware token modulation strategy that integrates scene-level reasoning with token-level adjustment to adapt refusals according to the predicted safety verdict. Extensive experiments across diverse MLLM architectures and safety benchmarks, covering undersensitivity, oversensitivity, and general safety evaluations, show that SafeCoDe consistently improves context-sensitive refusal behaviors while preserving model helpfulness.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
