MPruner：基于CKA互信息剪枝优化神经网络规模
MPruner: Optimizing Neural Network Size with CKA-Based Mutual Information Pruning
AI总结：
本文提出MPruner剪枝算法，利用CKA相似性度量的层聚类引入全局互信息，实现更精确的逐层剪枝，在CNN和Transformer模型上实现高达50%的参数与内存削减且几乎不损失精度。
摘要：
确定神经网络的最优规模至关重要，因为它直接影响运行时性能和内存使用。剪枝是一种成熟的模型压缩技术，能够在数学上保证精度保持的同时减小神经网络规模。然而，许多近期剪枝方法忽视了单个模型组件的全局贡献，使得难以确保剪枝后的模型满足期望的数据集和性能要求。为应对这些挑战，我们开发了一种新的剪枝算法MPruner，该算法通过向量相似性利用互信息。MPruner利用基于中心核对齐（CKA）相似性度量的层聚类，使我们能够纳入神经网络的全局信息，从而实现更精确、更高效的逐层剪枝。我们在多种架构和配置上评估了MPruner，展示了其通用性并提供了实用指南。对于CNN和基于transformer的模型，MPruner在参数和内存使用上实现了高达50%的削减，且精度损失极小甚至无损失。
英文摘要：
Determining the optimal size of a neural network is critical, as it directly impacts runtime performance and memory usage. Pruning is a well-established model compression technique that reduces the size of neural networks while mathematically guaranteeing accuracy preservation. However, many recent pruning methods overlook the global contributions of individual model components, making it difficult to ensure that a pruned model meets the desired dataset and performance requirements. To address these challenges, we developed a new pruning algorithm, MPruner, that leverages mutual information through vector similarity. MPruner utilizes layer clustering with the Centered Kernel Alignment (CKA) similarity metric, allowing us to incorporate global information from the neural network for more precise and efficient layer-wise pruning. We evaluated MPruner across various architectures and configurations, demonstrating its versatility and providing practical guidelines. MPruner achieved up to a 50% reduction in parameters and memory usage for CNN and transformer-based models, with minimal to no loss in accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
