利用输入自适应优化实现视觉Transformer的对抗性效率退化研究综述
A Survey of Adversarial Efficiency Degradation for Vision Transformer by Exploiting Input-adaptive Optimization
查看机构详情
- Indian Institute of Technology Guwahati（印度古瓦哈蒂理工学院）
- Nanyang Technological University（南洋理工大学）
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本综述针对视觉Transformer的输入自适应推理机制，研究SlowFormer和DeSparsify等对抗性效率退化攻击，分析其攻击面、脆弱优化及防御措施，为设计轻量防御提供依据。
中文摘要 AI 辅助
视觉Transformer（ViT）越来越依赖输入自适应推理，例如token剪枝和提前停止，以满足能耗和延迟预算。本综述研究一类针对这些机制的新型对抗性效率退化攻击，这类攻击会增加计算量但不一定降低准确率。我们在A-ViT、ATS和AdaViT这三个流行的token剪枝框架上，对两种代表性攻击SlowFormer（一种通用对抗性补丁）和DeSparsify（每张图像的扰动）进行了统一和比较。我们使用GFLOPs、准确率损失和攻击成功率（AS，衡量攻击剥夺模型计算节省的程度）来标准化报告。理解这些攻击对于设计既能缓解风险又保持轻量的对策至关重要，因为部署通常发生在移动设备或嵌入式设备等低功耗场景中。为组织分析，我们聚焦三个问题：输入自适应优化（如token剪枝和提前停止）如何为效率退化创造攻击面；这类攻击实际如何运作以及哪些优化最易受攻击；当前存在哪些防御措施以及它们是否能在攻击下有效恢复效率。
英文摘要
Vision Transformers (ViTs) increasingly rely on input-adaptive inference, such as token pruning and early halting, to meet energy and latency budgets. This survey examines a recent class of adversarial efficiency degradation attacks that target these mechanisms to increase computation without necessarily degrading accuracy. We unify and compare two representative attacks, SlowFormer (a universal adversarial patch) and DeSparsify (per-image perturbations), across three popular token-pruning frameworks: A-ViT, ATS, and AdaViT. We standardize reporting using GFLOPs, accuracy loss, and an Attack Success (AS) metric that measures how much of the model's compute savings the attack takes away. Understanding these attacks is crucial for designing countermeasures that not only mitigate risk but also remain lightweight, since deployment often occurs in low-power settings such as mobile or embedded devices. To organize our analysis, we focus on three questions: how input-adaptive optimizations (e.g., token pruning and early halting) create attack surfaces for efficiency degradation; how such attacks operate in practice and which optimizations are most vulnerable; and which defenses exist today and whether they meaningfully restore efficiency under attack.
