GuardTrace-VL: 通过迭代安全监督检测不安全的多模型推理
GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision
AI总结：
GuardTrace-VL通过联合图像-文本分析监控多模态推理全过程，有效检测不安全推理内容，实现93.1%的F1分数提升。
摘要：
多模态大推理模型（MLRMs）正越来越多地用于生成显式中间推理过程的视觉-语言任务。然而，即使最终答案无害，推理轨迹也可能包含不安全内容，这会带来部署风险。现有的多模态安全防护主要只评估输入问题和最终答案，忽略了中间推理过程。这种忽视允许在推理过程中出现未被检测到的危害，例如偏见推断或违反政策的视觉上下文使用。我们引入GuardTrace-VL，一种视觉感知的安全审计员，通过联合图像-文本分析监控完整的Question-Thinking-Answer（QTA）流程，使在推理阶段能够检测到不安全内容。为了支持训练和评估，我们构建了GuardTrace数据集，该数据集通过多样的提示策略生成，并通过MLRM和人工的投票和验证流程进行精炼。此外，我们提出了一种三阶段渐进训练方案，结合数据精炼过程，使模型能够根据不同的风险级别学习细腻且上下文相关的安全偏好。在我们提出的涵盖领域内和领域外场景的测试集上，GuardTrace-VL模型在不安全推理检测任务上实现了93.1%的F1分数，比之前最强的多模态安全防御方法在F1分数上提高了13.5%。代码将公开发布。
英文摘要：
Multimodal large reasoning models (MLRMs) are increasingly deployed for vision-language tasks that produce explicit intermediate rationales. However, reasoning traces can contain unsafe content even when the final answer is non-harmful, creating deployment risks. Existing multimodal safety guards primarily evaluate only the input question and the final answer, neglecting the intermediate reasoning process. This oversight allows undetected harm, such as biased inferences or policy-violating use of visual context, to emerge during reasoning. We introduce GuardTrace-VL, a vision-aware safety auditor that monitors the full Question-Thinking-Answer (QTA) pipeline via joint image-text analysis, enabling detection of unsafe content as it emerges in the reasoning stage. To support training and evaluation, we construct the GuardTrace dataset, which is generated through diverse prompting strategies and refined via a MLRM- and human-based voting and verification pipeline. Furthermore, we propose a three-stage progressive training scheme combined with the data refinement process, enabling the model to learn nuanced and context-dependent safety preferences according to different risk levels. On our proposed test set covering both in-domain and out-of-domain scenarios, GuardTrace-VL model achieves an F1 score of 93.1% on unsafe reasoning detection tasks, representing a 13.5% improvement in F1 score compared to the previous strongest multimodal safety defense methods. The codes will be made publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
