SaFeR-ToolKit: 通过虚拟工具调用实现多模态安全的结构化推理
SaFeR-ToolKit: Structured Reasoning via Virtual Tool Calling for Multimodal Safety
AI总结:
SaFeR-ToolKit通过虚拟工具调用实现多模态安全的结构化推理,提升安全性、帮助性和推理严谨性,同时保持通用能力。
摘要:
视觉-语言模型仍然容易受到多模态劫持和过度拒绝的影响,因为安全性取决于视觉证据和用户意图,而许多对齐流程只监督最终响应。为了解决这个问题,我们提出了SaFeR-ToolKit,它将安全性决策制定正式化为可检查的协议。具体来说,规划器指定一个角色,一个感知→推理→决策的工具集,以及一个受约束的转换图,而响应者在最终答案之前输出一个带类型的关键值工具轨迹。为了使该协议在实践中可靠地执行,我们训练了一个单一策略,采用三阶段课程(SFT→DPO→GRPO),其中GRPO直接监督工具使用,而不仅仅是回答层面的反馈。我们的贡献是双方面的:I. 数据集。第一个基于工具的安全推理数据集,包含31,654个示例(SFT 6k,DPO 18.6k,GRPO 6k)加上1k个保留评估示例。II. 实验。在Qwen2.5-VL上,SaFeR-ToolKit显著提高了安全性/帮助性/推理严谨性在3B(29.39/45.04/4.98→84.40/71.13/78.87)和7B(53.21/52.92/19.26→86.34/80.79/85.34),同时保持了通用能力(3B:58.67→59.21;7B:66.39→66.81)。代码可在https://github.com/Duebassx/SaFeR_ToolKit上获得。
英文摘要:
Vision-language models remain susceptible to multimodal jailbreaks and over-refusal because safety hinges on both visual evidence and user intent, while many alignment pipelines supervise only the final response. To address this, we present SaFeR-ToolKit, which formalizes safety decision-making as a checkable protocol. Concretely, a planner specifies a persona, a Perception $\to$ Reasoning $\to$ Decision tool set, and a constrained transition graph, while a responder outputs a typed key-value tool trace before the final answer. To make the protocol reliably followed in practice, we train a single policy with a three-stage curriculum (SFT $\to$ DPO $\to$ GRPO), where GRPO directly supervises tool usage beyond answer-level feedback. Our contributions are two-fold: I. Dataset. The first tool-based safety reasoning dataset, comprising 31,654 examples (SFT 6k, DPO 18.6k, GRPO 6k) plus 1k held-out evaluation. II. Experiments. On Qwen2.5-VL, SaFeR-ToolKit significantly improves Safety/Helpfulness/Reasoning Rigor on 3B (29.39/45.04/4.98 $\to$ 84.40/71.13/78.87) and 7B (53.21/52.92/19.26 $\to$ 86.34/80.79/85.34), while preserving general capabilities (3B: 58.67 $\to$ 59.21; 7B: 66.39 $\to$ 66.81). Codes are available at https://github.com/Duebassx/SaFeR_ToolKit.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。