HASTE: 一种无需训练、动态且可引导的预训练卷积神经网络压缩框架
HASTE: A Framework for Training-Free, Dynamic, and Steerable Compression of Pre-Trained Convolutional Neural Networks
AI总结：
提出HASTE模块，利用局部敏感哈希在推理时动态合并冗余通道，实现预训练CNN的无训练压缩，在CIFAR-10上ResNet34减少46.2% FLOPs且精度仅降1.25%。
摘要：
在资源受限设备上部署大型卷积神经网络（CNN）因其高计算成本而具有挑战性。尽管动态执行方法前景广阔，但现有的CNN方法通常需要专门的训练或微调，这限制了它们在预训练模型上的有效性，并且需要数据访问。为了解决这一差距，我们提出了HASTE（可处理效率的哈希），这是一个即插即用的卷积模块，能够实现对大型预训练CNN的无需训练的动态压缩。在推理时，HASTE使用局部敏感哈希来识别并合并潜在特征图的冗余通道（基于块的方式）。该过程同时压缩输入特征及其对应滤波器的深度，从而产生计算成本更低的卷积。我们在CIFAR-10和ImageNet上跨多种架构进行了大量实验，结果表明，在CIFAR-10上，ResNet34的FLOPs减少了46.2%，而准确率仅下降1.25%，且无需重新训练。我们通过全面的消融研究来验证核心设计选择，分析方法的特性和局限性，并讨论我们的通道合并方案与Vision Transformers中概念相关的令牌合并任务之间的联系，从而支持我们的主张。我们的结果表明，HASTE为预训练CNN在运行时提供了一种可引导压缩的有效解决方案，为高效深度学习方法的部署开辟了新的可能性。
英文摘要：
Deploying large convolutional neural networks (CNNs) on resource-constrained devices is challenging due to their high computational cost. While dynamic execution methods are promising, existing approaches for CNNs typically require specialized training or fine-tuning, limiting their effectiveness when applied to pre-trained models and requiring data access. To address this gap, we propose HASTE (Hashing for Tractable Efficiency), a plug-and-play convolution module that enables training-free, dynamic compression of large pre-trained CNNs. At inference time, HASTE uses locality-sensitive hashing to identify and merge redundant channels of latent feature maps on a patch-wise basis. This process simultaneously compresses the depth of both input features and their corresponding filters, resulting in computationally cheaper convolutions. We conduct extensive experiments on CIFAR-10 and ImageNet across a range of architectures, demonstrating a 46.2% FLOPs reduction in a ResNet34 on CIFAR-10 with only a 1.25% drop in accuracy, without any retraining. We support our claims by comprehensive ablation studies to validate our core design choices, an analysis of the method's properties and limitations, and a discussion that connects our channel merging scheme to the conceptually related task of token merging in Vision Transformers. Our results demonstrate that HASTE provides an effective solution for steerable compression of pre-trained CNNs at runtime, opening new possibilities for the deployment of efficient deep learning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
