剪枝一个额外令牌就够了：利用延迟-工作负载非线性实现边缘视觉Transformer
Pruning One More Token is Enough: Leveraging Latency-Workload Non-Linearities for Vision Transformers on the Edge
AI总结：
本文针对边缘设备小负载场景，利用延迟-工作负载非线性关系确定令牌剪枝调度，提出免训练的令牌剪枝方法，在相似延迟下实现更高精度。
摘要：
本文研究如何在边缘设备上高效部署视觉Transformer以处理小规模工作负载。近期方法通过移除或合并令牌来降低Transformer神经网络的延迟，同时仅造成较小的精度损失。然而，这些方法并未针对边缘设备部署进行设计：它们未利用延迟-工作负载趋势信息来提升效率。我们针对这一不足展开研究。首先，我们识别影响ViT延迟-工作负载关系的因素。其次，我们利用非线性的延迟-工作负载关系确定令牌剪枝调度。第三，我们展示了一种利用该调度的免训练令牌剪枝方法。我们表明，其他方法可能将延迟增加2%-30%，而我们将延迟降低9%-26%。在相似延迟（设备间差异在5.2%或7毫秒以内）下，我们达到78.6%-84.5%的ImageNet1K精度，而最先进的Token Merging方法仅达到45.8%-85.4%。
英文摘要：
This paper investigates how to efficiently deploy vision transformers on edge devices for small workloads. Recent methods reduce the latency of transformer neural networks by removing or merging tokens, with small accuracy degradation. However, these methods are not designed with edge device deployment in mind: they do not leverage information about the latency-workload trends to improve efficiency. We address this shortcoming in our work. First, we identify factors that affect ViT latency-workload relationships. Second, we determine token pruning schedule by leveraging non-linear latency-workload relationships. Third, we demonstrate a training-free, token pruning method utilizing this schedule. We show other methods may increase latency by 2-30%, while we reduce latency by 9-26%. For similar latency (within 5.2% or 7ms) across devices we achieve 78.6%-84.5% ImageNet1K accuracy, while the state-of-the-art, Token Merging, achieves 45.8%-85.4%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
