arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.05258cs.CV
收藏

网络宽度搜索的局部自由权值共享

Locally Free Weight Sharing for Network Width Search

The University of Sydney(悉尼大学);SenseTime Research(商汤科技研究院);Institute for Artificial Intelligence, Tsinghua University(清华大学人工智能研究院)

Xiu Su, Shan You, Tao Huang, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu

摘要:

搜索网络宽度是用硬件预算来精简深度神经网络的有效方法。为此,通常利用单次超网作为性能评估器来对性能\wrt ~不同宽度进行排名。然而,目前的方法主要遵循人工固定的权重分配模式,这是有限的区分不同宽度的性能差距。在本文中,为了更好地评估每个宽度,我们相应地提出了一个局部自由的权重共享策略(CafeNet)。在CafeNet中,权重被更自由地共享,并且每个宽度由其基本通道和自由通道联合指示,其中自由通道应该位于局部区域中以更好地表示每个宽度。此外,我们建议进一步减少搜索空间,利用我们引入的FLOPS敏感的bin。因此,我们的CafeNet可以随机训练,并在min-min策略内进行优化。在ImageNet、CIFAR-10、CelebA和MS COCO数据集上进行的大量实验验证了我们与其他最先进的基线相比的优越性。例如,我们的方法可以通过更精细地搜索其宽度,将基准NAS网络EfficientNet-B 0进一步提高0.41\%。

英文摘要:

Searching for network width is an effective way to slim deep neural networks with hardware budgets. With this aim, a one-shot supernet is usually leveraged as a performance evaluator to rank the performance \wrt~different width. Nevertheless, current methods mainly follow a manually fixed weight sharing pattern, which is limited to distinguish the performance gap of different width. In this paper, to better evaluate each width, we propose a locally free weight sharing strategy (CafeNet) accordingly. In CafeNet, weights are more freely shared, and each width is jointly indicated by its base channels and free channels, where free channels are supposed to loCAte FrEely in a local zone to better represent each width. Besides, we propose to further reduce the search space by leveraging our introduced FLOPs-sensitive bins. As a result, our CafeNet can be trained stochastically and get optimized within a min-min strategy. Extensive experiments on ImageNet, CIFAR-10, CelebA and MS COCO dataset have verified our superiority comparing to other state-of-the-art baselines. For example, our method can further boost the benchmark NAS network EfficientNet-B0 by 0.41\% via searching its width more delicately.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑