arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.23960cs.CVcs.AIcs.CR
收藏

SafeVision:具备鲁棒策略遵循性与可解释性的高效图像护栏

SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainability

University of Chicago(芝加哥大学);Virtue AI;Meta;University of Wisconsin, Madison(威斯康星大学麦迪逊分校)

Peiyang Xu, Minzhou Pan, Zhaorun Chen, Shuang Yang, Chaowei Xiao, Bo Li

AI总结:

针对传统图像护栏模型误分类多、难适配新威胁的问题,提出融合类人推理的SafeVision图像护栏,搭配新数据集VisionHarm,实现了更优的检测性能与更快的推理速度,可动态适配新安全策略。

摘要:

随着数字媒体的快速普及,针对不安全内容构建高效且透明的防护机制的需求比以往任何时候都更为迫切。传统图像护栏模型受限于预定义类别,由于仅采用基于特征的学习方式、缺乏语义推理,经常会对内容出现误分类。此外,这类模型难以适应新出现的威胁,应对新威胁需要成本高昂的重新训练。为解决这些局限,我们提出了SafeVision——一种新型图像护栏,它融合类人推理能力以提升适应性与透明度。我们的方法包含一套高效的数据收集与生成框架、一条遵循策略的训练流水线以及一个定制化损失函数。我们还提出了多样化的QA生成与训练策略,以提升学习效果。SafeVision在推理阶段可动态匹配不断演进的安全策略,无需重新训练,同时能实现精准的风险评估与解释。鉴于现有不安全图像基准存在要么缺乏细粒度、要么覆盖风险类型有限的局限,我们推出了VisionHarm数据集,这是一个包含两个子集的高质量数据集:VisionHarm第三方子集(VisionHarm-T)与VisionHarm综合子集(VisionHarm-C),覆盖多种有害类别。通过大量实验,我们证明SafeVision在不同基准上均取得了当前最优性能。SafeVision在VisionHarm-T上的表现优于GPT-4o 8.6%,在VisionHarm-C上优于GPT-4o 15.5%,同时速度快16倍以上。SafeVision构建了一套全面、遵循策略且可解释的图像护栏,能够动态适应新出现的威胁。

英文摘要:

With the rapid proliferation of digital media, the need for efficient and transparent safeguards against unsafe content is more critical than ever. Traditional image guardrail models, constrained by predefined categories, often misclassify content due to their pure feature-based learning without semantic reasoning. Moreover, these models struggle to adapt to emerging threats, requiring costly retraining for new threats. To address these limitations, we introduce SafeVision, a novel image guardrail that integrates human-like reasoning to enhance adaptability and transparency. Our approach incorporates an effective data collection and generation framework, a policy-following training pipeline, and a customized loss function. We also propose a diverse QA generation and training strategy to enhance learning effectiveness. SafeVision dynamically aligns with evolving safety policies at inference time, eliminating the need for retraining while ensuring precise risk assessments and explanations. Recognizing the limitations of existing unsafe image benchmarks, which either lack granularity or cover limited risks, we introduce VisionHarm, a high-quality dataset comprising two subsets: VisionHarm Third-party (VisionHarm-T) and VisionHarm Comprehensive(VisionHarm-C), spanning diverse harmful categories. Through extensive experiments, we show that SafeVision achieves state-of-the-art performance on different benchmarks. SafeVision outperforms GPT-4o by 8.6% on VisionHarm-T and by 15.5% on VisionHarm-C, while being over 16x faster. SafeVision sets a comprehensive, policy-following, and explainable image guardrail with dynamic adaptation to emerging threats.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑