AI 中文总结
研究多模态大语言模型的攻击问题，提出FlowGuard框架，通过监测内部多模态一致性检测有害输入，受部分信息分解启发得出FlowVectors量化相关指标，有效降低攻击成功率且减少效用损失和延迟，为多模态推理提供有效防御。
AI 中文摘要
多模态大语言模型引入了单模态系统中不存在的攻击面，对手可跨模态分布恶意意图以规避单模态防护。因此应使用跨模态一致性作为检测信号。良性输入会诱导仅文本和仅视觉推理产生兼容的预测行为，融合时稳定，而对抗性操纵会破坏这种一致性。现有防御忽略内部融合过程，脆弱且计算昂贵。我们提出FlowGuard，通过监测内部多模态一致性检测有害输入。它受部分信息分解启发得出FlowVectors，量化跨模态冗余等。在仅基于良性数据训练的单类分类问题中，FlowGuard将未见攻击的攻击成功率从>90%降至<15%，效用损失<3%，延迟最多降低6倍。结果表明监测跨模态一致性为多模态推理提供了有效防御。
英文摘要
Multimodal large language models introduce attack surfaces absent in unimodal systems: adversaries can distribute malicious intent across modalities to evade unimodal safeguards. This motivates using cross-modal consistency as a detection signal rather than inspecting each modality in isolation. Our key observation is that benign inputs induce compatible predictive behavior from text-only and vision-only reasoning that stabilizes when fused, whereas adversarial manipulation disrupts this consistency, causing abnormal multimodal behavior. Existing defenses that examine raw inputs or outputs overlook this internal fusion process, rendering them brittle and computationally expensive. We propose FlowGuard, a lightweight inference-time framework that detects harmful inputs by monitoring internal multimodal consistency. Unlike approaches that rely on scalar confidence metrics, FlowGuard derives FlowVectors inspired by Partial Information Decomposition that quantify cross-modal redundancy, synergy, and modality-specific dominance, capturing whether fused multimodal predictions remain aligned with unimodal semantic evidence. In a one-class classification problem trained solely on benign data, FlowGuard reduces Attack Success Rates from >90% to <15% on unseen attacks, with <3% utility loss and up to a 6 times latency reduction. Our results demonstrate that monitoring cross-modal consistency offers an efficient and effective defense for multimodal reasoning.
CommentsAccepted as Spotlight Paper at ICML 2026
