arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1809.05262cs.LGcs.CVstat.ML
收藏

网络重铸:一种通用的网络结构转换方法

Network Recasting: A Universal Method for Network Architecture Transformation

Seoul National University(首尔国立大学);Neural Processing Research Center (NPRC)(神经处理研究中心(NPRC))

Joonsang Yu, Sungbum Kang, Kiyoung Choi

摘要:

本文提出了网络重铸作为一种通用的方法,网络体系结构的转变。这种方法的主要目标是通过转换来加速推理过程,但也可以有许多其他的实际应用。该方法基于逐块重铸;它将预先训练的教师网络中的每个源块重铸到学生网络中的目标块。对于重铸,目标块被训练成使得其输出激活近似于源块的输出激活。这种以顺序方式的逐块重铸在保持准确性的同时变换了网络架构。此方法可用于将任意教师网络类型转换为任意学生网络类型。它甚至可以生成由两种或多种类型的块组成的混合架构网络。网络重铸可以生成具有更少参数和/或激活的网络,这显著减少了推理时间。当然,它可以用于网络压缩,通过将训练过的网络重铸为相同类型的较小网络。我们的实验表明,它优于以前的压缩方法在GPU上的实际加速。

英文摘要:

This paper proposes network recasting as a general method for network architecture transformation. The primary goal of this method is to accelerate the inference process through the transformation, but there can be many other practical applications. The method is based on block-wise recasting; it recasts each source block in a pre-trained teacher network to a target block in a student network. For the recasting, a target block is trained such that its output activation approximates that of the source block. Such a block-by-block recasting in a sequential manner transforms the network architecture while preserving the accuracy. This method can be used to transform an arbitrary teacher network type to an arbitrary student network type. It can even generate a mixed-architecture network that consists of two or more types of block. The network recasting can generate a network with fewer parameters and/or activations, which reduce the inference time significantly. Naturally, it can be used for network compression by recasting a trained network into a smaller network of the same type. Our experiments show that it outperforms previous compression approaches in terms of actual speedup on a GPU.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑