通过因果发现和双模态安全子空间投影诊断和修复视觉-语言模型中的不安全通道
Diagnosing and Repairing Unsafe Channels in Vision-Language Models via Causal Discovery and Dual-Modal Safety Subspace Projection
AI总结：
本文提出CARE框架，通过因果分析和双模态安全子空间投影修复视觉-语言模型中的不安全通道，提升安全性而不影响多模态能力。
摘要：
大型视觉-语言模型（LVLMs）在多模态理解和推理任务中表现出色，但其内部安全机制仍不透明且控制不足。本文提出CARE框架，通过因果中介分析识别导致不安全行为的神经元和层，引入双模态安全子空间投影方法，通过良性与恶意激活之间的广义特征分解学习视觉和文本模态的通用安全子空间。在推理过程中，通过混合融合机制动态将激活投影到这些安全子空间，平衡视觉和文本修正，有效抑制不安全特征并保持语义保真度。在多个安全基准测试中，我们的因果-子空间修复框架显著提高了安全性鲁棒性，不降低多模态能力，优于先前的激活引导和对齐基线。此外，该方法具有良好的可迁移性，能防御未见过的攻击。
英文摘要：
Large Vision-Language Models (LVLMs) have achieved impressive performance across multimodal understanding and reasoning tasks, yet their internal safety mechanisms remain opaque and poorly controlled. In this work, we present a comprehensive framework for diagnosing and repairing unsafe channels within LVLMs (CARE). We first perform causal mediation analysis to identify neurons and layers that are causally responsible for unsafe behaviors. Based on these findings, we introduce a dual-modal safety subspace projection method that learns generalized safety subspaces for both visual and textual modalities through generalized eigen-decomposition between benign and malicious activations. During inference, activations are dynamically projected toward these safety subspaces via a hybrid fusion mechanism that adaptively balances visual and textual corrections, effectively suppressing unsafe features while preserving semantic fidelity. Extensive experiments on multiple safety benchmarks demonstrate that our causal-subspace repair framework significantly enhances safety robustness without degrading general multimodal capabilities, outperforming prior activation steering and alignment-based baselines. Additionally, our method exhibits good transferability, defending against unseen attacks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
