LLaVAShield: 保障视觉语言模型中的多模态多轮对话安全
LLaVAShield: Safeguarding Multimodal Multi-Turn Dialogues in Vision-Language Models
AI总结：
LLaVAShield通过构建MMDS数据集和MMRT框架，有效提升多模态多轮对话的安全性，优于现有VLMs和内容审查工具，揭示了主流模型的安全漏洞。
摘要：
随着视觉语言模型（VLMs）向交互式、多轮使用发展，多模态多轮对话的安全问题变得更加严峻。这种对话以隐藏恶意意图、上下文风险累积和跨模态联合风险为特征，这些特征限制了针对单轮或单模态设置设计的内容审查方法的有效性。为了解决这些限制，我们首先构建了多模态多轮对话安全（MMDS）数据集，包含4,484个注释对话和一个包含8个主要类别和60个子类别的全面风险分类体系。作为MMDS构建的一部分，我们引入了多模态多轮红队（MMRT），这是一种自动框架，用于生成不安全的多模态多轮对话。我们进一步提出了LLaVAShield，它在多模态多轮对话中根据指定的政策维度审核用户输入和助手响应的安全性。广泛的实验表明，LLaVAShield在性能上显著优于最先进的VLMs和现有内容审查工具，同时表现出强大的泛化能力和灵活的政策适应性。此外，我们分析了主流VLMs对有害输入的漏洞，并评估了关键组件的贡献，推进了对多模态多轮对话安全机制的理解。
英文摘要：
As Vision-Language Models (VLMs) move into interactive, multi-turn use, safety concerns intensify for multimodal multi-turn dialogue, which is characterized by concealment of malicious intent, contextual risk accumulation, and cross-modal joint risk. These characteristics limit the effectiveness of content moderation approaches designed for single-turn or single-modality settings. To address these limitations, we first construct the Multimodal Multi-turn Dialogue Safety (MMDS) dataset, comprising 4,484 annotated dialogues and a comprehensive risk taxonomy with 8 primary and 60 subdimensions. As part of MMDS construction, we introduce Multimodal Multi-turn Red Teaming (MMRT), an automated framework for generating unsafe multimodal multi-turn dialogues. We further propose LLaVAShield, which audits the safety of both user inputs and assistant responses under specified policy dimensions in multimodal multi-turn dialogues. Extensive experiments show that LLaVAShield significantly outperforms state-of-the-art VLMs and existing content moderation tools while demonstrating strong generalization and flexible policy adaptation. Additionally, we analyze vulnerabilities of mainstream VLMs to harmful inputs and evaluate the contribution of key components, advancing understanding of safety mechanisms in multimodal multi-turn dialogues.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
