PrimeSVT: 一种具有优先压缩策略的自动化内存感知剪枝框架用于脉冲视觉Transformer
PrimeSVT: An Automated Memory-aware Pruning Framework with Prioritized Compression Policy for Spiking Vision Transformers
AI总结：
提出PrimeSVT框架，通过自动化结构化剪枝和优先压缩策略，在满足精度和内存约束下压缩脉冲视觉Transformer，实现内存节省26.68%且精度损失小于3%。
摘要：
脉冲视觉Transformer（SViT）的大尺寸仍然阻碍其嵌入式实现，因此需要模型压缩。现有工作通过非结构化剪枝压缩SViT模型，这需要专门的硬件加速器来利用其特定的稀疏模式以最大化效率提升。此外，它们的手动方法需要大量设计时间来为每个网络找到合适的剪枝设置，因此这种方法不可扩展。为了解决这一限制，我们提出了PrimeSVT，一种新颖的框架，对预训练的SViT模型执行自动化的内存感知结构化剪枝，从而在推理期间最大化其效率提升，适用于广泛使用的计算架构。为此，PrimeSVT首先根据层的大小（即参数数量）对SViT层进行排序，根据它们在不同剪枝率下的鲁棒性识别目标剪枝层，然后利用这个顺序从最大层到最小层逐层顺序压缩模型（即所谓的优先压缩策略），同时考虑用户定义的约束（即可接受的精度和内存节省）。在每一层中，PrimeSVT基于L2范数值采用通道级滤波器剪枝，以结构性地移除不重要的权重。实验结果表明，PrimeSVT通过自动化单次剪枝节省了26.68%的内存，同时将精度保持在原始未剪枝SViT模型（73.3%）的3%以内（未微调时为70.3%，微调后为72.9%），从而满足了精度和内存约束。这些表明我们的PrimeSVT框架实现了SViT及其嵌入式实现的设计自动化。
英文摘要：
The large sizes of Spiking Vision Transformers (SViTs) still hinder their embedded implementation, highlighting the need for model compression. State-of-the-art works compress SViT models through unstructured pruning, which needs specialized hardware accelerators for their specific sparsity patterns to maximize efficiency gains. Moreover, their manual approach requires a huge design time to find an appropriate pruning setting for each network, thus making this approach not scalable. To address this limitation, we propose PrimeSVT, a novel framework that performs automated memory-aware structured pruning on pre-trained SViT models, thereby maximizing their efficiency gains during inference amenable to widely-used computing architectures. To achieve this, PrimeSVT first sorts the SViT layers based on their sizes (i.e., number of parameters), identifies the targeted pruning layers based on their robustness under different pruning rates, then leverages this order for compressing the model layer-by-layer sequentially from the largest one to the smallest one (i.e., so-called prioritized compression policy), while considering the user-defined constraints (i.e., acceptable accuracy and memory saving). In each layer, PrimeSVT employs channel-wise filter pruning based on their L2-norm values to structurally remove the non-significant weights. Experimental results show that PrimeSVT saves 26.68% memory through automated single-shot pruning, while preserving accuracy within 3% (70.3% without fine-tuning and 72.9% with fine-tuning) from the original unpruned SViT model (73.3%), thus meeting the accuracy and memory constraints. These show that our PrimeSVT framework enables design automation for SViTs and their embedded implementation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
