针对VQA-NLE的对抗攻击：揭示并缓解视觉问答解释中的不一致性
Adversarial Attacks on VQA-NLE: Exposing and Alleviating Inconsistencies in Visual Question Answering Explanations
AI总结：
本文针对VQA-NLE系统提出图像扰动与问题扰动两种对抗攻击以暴露解释不一致性，并引入外部知识缓解方法提升模型鲁棒性，在标准基准上验证了攻击有效性与防御潜力。
摘要：
视觉问答中的自然语言解释（VQA-NLE）旨在通过阐明黑盒模型的决策过程使其更加透明。然而，我们发现现有的VQA-NLE系统可能产生不一致的解释，并在没有真正理解底层上下文的情况下得出结论，暴露出其推理流程或解释生成机制中的弱点。为了突出这些脆弱性，我们不仅利用现有的对抗策略来扰动问题，还提出了一种新颖的策略，通过最小限度地修改图像来诱导矛盾或虚假的输出。我们进一步引入了一种缓解方法，利用外部知识来减轻这些不一致性，从而增强模型的鲁棒性。在两个标准基准和两个广泛使用的VQA-NLE模型上的广泛评估凸显了我们攻击的有效性以及基于知识的防御的潜力，最终揭示了当前VQA-NLE系统中紧迫的安全性和可靠性问题。
英文摘要：
Natural language explanations in visual question answering (VQA-NLE) aim to make black-box models more transparent by elucidating their decision-making processes. However, we find that existing VQA-NLE systems can produce inconsistent explanations and reach conclusions without genuinely understanding the underlying context, exposing weaknesses in either their inference pipeline or explanation-generation mechanism. To highlight these vulnerabilities, we not only leverage an existing adversarial strategy to perturb questions but also propose a novel strategy that minimally alters images to induce contradictory or spurious outputs. We further introduce a mitigation method that leverages external knowledge to alleviate these inconsistencies, thereby bolstering model robustness. Extensive evaluations on two standard benchmarks and two widely used VQA-NLE models underscore the effectiveness of our attacks and the potential of knowledge-based defenses, ultimately revealing pressing security and reliability concerns in current VQA-NLE systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
