arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.07086cs.CVcs.LG
收藏

任务相关性并非局部可替换性:通道信息的双轴视角

Task Relevance Is Not Local Replaceability: A Two-Axis View of Channel Information

Kempner Institute for the Study of Natural and Artificial Intelligence(自然与人工智能研究学院);Harvard University(哈佛大学);Department of Neurobiology, Howard Hughes Medical Institute(神经生物学系,霍华德·休斯医学研究所);Harvard Medical School(哈佛医学院)

Houman Safaai, Andrew T. Landau, Celia C. Beron, Yasin Mazloumi, Bernardo L. Sabatini

AI总结:

本文提出双轴视角分析通道重要性,区分任务相关性和局部可替换性,发现不同轴在不同网络中表现各异,局部轴更可靠预测可删除性。

摘要:

视觉网络中通道重要性通常通过单一评分总结,但此总结掩盖了两个问题:通道与任务的相关性以及在移除通道时同层其他通道能否替代其功能。我们引入双轴视角,分别衡量输入捕获和同层重叠(局部轴)以及任务信息和目标超额信息(目标轴)。在ResNet-18、VGG-16和MobileNetV2上训练的CIFAR-100数据集中,两个轴弱相关,导致不同的通道分组,并在训练过程中迅速分离,尽管在随机初始化时强相关。高斯线性分析解释了这种分离如何通过残差梯度方向产生,而病变加同层替代实验显示同层支持能细化可删除性,超越输入捕获和任务相关性。在固定FLOPs匹配剪枝协议下,局部轴指标在三个CIFAR-100架构中比目标轴指标更可靠预测可删除性,且在压力测试中方向一致。这些发现表明轴层面的区别而非通用剪枝评分排名:局部可替换性比任务相关性更可靠指导可删除性,而基于范数的基线在VGG-16等架构中仍具竞争力。基于相关性的评分询问通道对任务的说明,而剪枝询问当其他通道可用时网络是否仍需该通道。

英文摘要:

Channel importance in vision networks is usually summarized by a single score. That summary hides two different questions: how much a channel is related to the task, and whether its function can be supplied by same-layer peers when the channel is removed. We call the second property local replaceability. We introduce a two-axis view that separates these questions. The local axis measures input capture and peer overlap, while the target axis measures task information and target-excess information. Across ResNet-18, VGG-16, and MobileNetV2 trained on CIFAR-100, the two axes are weakly aligned, induce different channel groupings, and separate rapidly during training despite being strongly coupled at random initialization. A Gaussian linear analysis accounts for how this separation can arise through residualized gradient directions, and lesion plus peer-replacement experiments show that peer support refines removability beyond input capture and task relevance alone. Under the fixed FLOPs-matched pruning protocol, local-axis metrics are more reliable predictors of removability than target-axis metrics across the three CIFAR-100 backbones, with the same direction preserved in stress tests on CIFAR-10, Tiny-ImageNet, ImageNet-100, and a ConvNeXt-T/ImageNet-100 pilot. These findings identify an axis-level distinction rather than a universal ranking of pruning scores: local replaceability is a more reliable guide to removability than target relevance, while norm-based baselines remain competitive in architectures such as VGG-16. Relevance-based scores ask what a channel says about the task; pruning asks whether the network still needs that channel when its peers remain available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑