arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1806.02639cs.LGcs.AIstat.ML
收藏

用于高效架构搜索的路径级网络转换

Path-Level Network Transformation for Efficient Architecture Search

Shanghai Jiao Tong University(上海交通大学);Massachusetts Institute of Technology(麻省理工学院)

Han Cai, Jiacheng Yang, Weinan Zhang, Song Han, Yong Yu

摘要:

我们引入了一个新的功能保持转换高效的神经结构搜索。这种网络转换允许重用先前训练的网络和现有的成功架构,从而提高样本效率。我们的目标是解决目前的网络转换操作的限制,只能执行层级架构的修改,如添加(修剪)过滤器或插入(删除)层,这无法改变连接路径的拓扑结构。我们提出的路径级变换操作使元控制器能够修改给定网络的路径拓扑,同时保持重用权重的优点,从而可以有效地设计具有复杂路径拓扑的有效结构,如Inception模型。我们进一步提出了一个双向树结构的强化学习元控制器,以探索一个简单而高度表达的树结构的架构空间,可以被视为一个泛化的多分支架构。我们在计算资源有限(约200 GPU小时)的图像分类数据集上进行了实验,我们观察到参数效率的提高和更好的测试结果(CIFAR-10上14.3M参数的测试准确率为97.70%,ImageNet在移动环境中的top-1准确率为74.6%),证明了我们设计的架构的有效性和可移植性。

英文摘要:

We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing successful architectures that improves sample efficiency. We aim to address the limitation of current network transformation operations that can only perform layer-level architecture modifications, such as adding (pruning) filters or inserting (removing) a layer, which fails to change the topology of connection paths. Our proposed path-level transformation operations enable the meta-controller to modify the path topology of the given network while keeping the merits of reusing weights, and thus allow efficiently designing effective structures with complex path topologies like Inception models. We further propose a bidirectional tree-structured reinforcement learning meta-controller to explore a simple yet highly expressive tree-structured architecture space that can be viewed as a generalization of multi-branch architectures. We experimented on the image classification datasets with limited computational resources (about 200 GPU-hours), where we observed improved parameter efficiency and better test results (97.70% test accuracy on CIFAR-10 with 14.3M parameters and 74.6% top-1 accuracy on ImageNet in the mobile setting), demonstrating the effectiveness and transferability of our designed architectures.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑