当表面欺骗：利用褶皱诱导的注意力转移攻击视觉-语言模型
When Surfaces Lie: Exploiting Wrinkle-Induced Attention Shift to Attack Vision-Language Models
AI总结：
本文提出一种参数化结构扰动方法，通过构建多尺度褶皱场和整合位移场扭曲与表面一致的外观变化，以攻击视觉-语言模型的鲁棒性。
摘要：
视觉-语言模型（VLMs）在各种任务中展示了卓越的跨模态理解能力，包括零样本分类、图像描述和视觉问答。然而，其对物理合理非刚性变形（如柔性表面的褶皱）的鲁棒性仍不明确。本文提出一种受三维织物褶皱力学启发的参数化结构扰动方法。具体而言，我们的方法通过构建多尺度褶皱场并整合位移场扭曲与表面一致的外观变化，生成逼真的非刚性扰动。为在视觉自然性和对抗有效性之间取得最佳平衡，我们设计了一个低维参数空间中的分层适应函数，并采用基于优化的搜索策略。我们使用两阶段框架评估我们的方法：扰动首先在零样本分类代理任务上优化，随后在生成任务上评估其可迁移性。实验结果表明，我们的方法显著降低了各种最新VLMs的性能，在图像描述和视觉问答任务中均优于基线。
英文摘要：
Visual-Language Models (VLMs) have demonstrated exceptional cross-modal understanding across various tasks, including zero-shot classification, image captioning, and visual question answering. However, their robustness to physically plausible non-rigid deformations-such as wrinkles on flexible surfaces-remains poorly understood. In this work, we propose a parametric structural perturbation method inspired by the mechanics of three-dimensional fabric wrinkles. Specifically, our method generates photorealistic non-rigid perturbations by constructing multi-scale wrinkle fields and integrating displacement field distortion with surface-consistent appearance variations. To achieve an optimal balance between visual naturalness and adversarial effectiveness, we design a hierarchical fitness function in a low-dimensional parameter space and employ an optimization-based search strategy. We evaluate our approach using a two-stage framework: perturbations are first optimized on a zero-shot classification proxy task and subsequently assessed for transferability on generative tasks. Experimental results demonstrate that our method significantly degrades the performance of various state-of-the-art VLMs, consistently outperforming baselines in both image captioning and visual question-answering tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
