CoDefend：基于扩散模型净化与提示优化的跨模态协同防御
CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimization
AI总结：
针对多模态大语言模型的对抗威胁，提出基于监督扩散去噪与提示优化的跨模态协同防御框架，显著提升了视觉模态的防御鲁棒性与对未知攻击的迁移性。
摘要：
多模态大型语言模型（MLLMs）通过整合视觉与文本模态，在图像描述、视觉问答及跨模态推理等任务中取得了显著成功。然而，其多模态特性也使其面临对抗性威胁，攻击者可扰动单一模态或联合扰动两者，以诱发有害、误导或违规的输出。现有防御策略存在明显局限：对抗训练通常仅提升对已知攻击的鲁棒性且计算成本高昂，而常规净化方法常导致图像质量下降，且对复杂多模态任务泛化不足。本文聚焦于防御视觉模态，该模态常作为对抗操作的主要入口。我们提出一种基于监督扩散的去噪框架，利用成对的对抗-干净图像数据集，通过定向、任务特定的引导来微调扩散模型。与DiffPure等先前的无监督净化方法不同，本方法实现了更高质量的重建，同时显著提升了多模态任务中的防御鲁棒性。此外，我们引入提示优化作为互补防御机制，增强了对多样及未知攻击策略的抵抗力。在图像描述与视觉问答上的大量实验表明，本方法不仅大幅提升了鲁棒性，还对未知对抗攻击展现出强迁移性。这些结果突显了监督扩散去噪在多模态防御中的有效性，为MLLMs在现实应用中更可靠、安全的部署铺平了道路。
英文摘要：
Multimodal Large Language Models (MLLMs) have achieved remarkable success in tasks such as image captioning, visual question answering, and cross-modal reasoning by integrating visual and textual modalities. However, their multimodal nature also exposes them to adversarial threats, where attackers can perturb either modality or both jointly to induce harmful, misleading, or policy violating outputs. Existing defense strategies, such as adversarial training and input purification, face notable limitations: adversarial training typically improves robustness only against known attacks while incurring high computational costs, whereas conventional purification approaches often suffer from degraded image quality and insufficient generalization to complex multimodal tasks. In this work, we focus on defending the visual modality, which frequently serves as the primary entry point for adversarial manipulation. We propose a supervised diffusion based denoising framework that leverages paired adversarial clean image datasets to fine-tune diffusion models with directional, task specific guidance. Unlike prior unsupervised purification methods such as DiffPure, our approach achieves higher quality reconstructions while significantly improving defense robustness in multimodal tasks. Furthermore, we incorporate prompt optimization as a complementary defense mechanism, enhancing resistance against diverse and unseen attack strategies. Extensive experiments on image captioning and visual question answering demonstrate that our method not only substantially improves robustness but also exhibits strong transferability to unknown adversarial attacks. These results highlight the effectiveness of supervised diffusion based denoising for multimodal defense, paving the way for more reliable and secure deployment of MLLMs in real world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
