arXiv 2103.119222021-03-22cs.CV

基于Monto-Carlo树搜索的优先级结构抽样

Prioritized Architecture Sampling with Monto-Carlo Tree Search

SenseTime Research（商汤科技研究院）

Xiu Su, Tao Huang, Yanxi Li, Shan You, Fei Wang, Chen Qian, Changshui Zhang, Chang Xu

备注：: Accepted by CVPR2021. We also release a NAS benchmark on the one-shot MobileNetV2 search space

摘要：

单次神经结构搜索（NAS）方法通过将整个搜索空间视为一个网络，只需要训练一次，从而显著降低了搜索成本。然而，当前的方法独立地选择每个操作而不考虑先前的层。此外，以巨大的计算代价获得的历史信息通常只使用一次，然后丢弃。在本文中，我们介绍了一种基于蒙特卡罗树搜索（MCTS）的抽样策略，其搜索空间被建模为蒙特卡罗树（MCT），它捕获层之间的依赖性。此外，中间结果存储在MCT中，以供未来决策和更好的勘探开发平衡。具体地说，MCT更新使用训练损失作为奖励的架构性能;为了准确地评估众多的节点，我们提出了节点通信和分层节点选择方法，分别在训练和搜索阶段，更好地利用操作奖励和层次信息。此外，为了公平地比较不同的NAS方法，我们构建了一个开源的NAS基准的宏搜索空间评估CIFAR-10，即NAS-Bench-Macro。在NAS-Bench-Macro和ImageNet上的大量实验表明，我们的方法显着提高了搜索效率和性能。例如，通过仅搜索$20$架构，我们获得的架构在ImageNet上以4.42亿FLOP实现了$78.0\%$ top-1的准确度。代码（基准）可在：\url{https://github.com/xiusu/NAS-Bench-Macro} .

英文摘要：

One-shot neural architecture search (NAS) methods significantly reduce the search cost by considering the whole search space as one network, which only needs to be trained once. However, current methods select each operation independently without considering previous layers. Besides, the historical information obtained with huge computation cost is usually used only once and then discarded. In this paper, we introduce a sampling strategy based on Monte Carlo tree search (MCTS) with the search space modeled as a Monte Carlo tree (MCT), which captures the dependency among layers. Furthermore, intermediate results are stored in the MCT for the future decision and a better exploration-exploitation balance. Concretely, MCT is updated using the training loss as a reward to the architecture performance; for accurately evaluating the numerous nodes, we propose node communication and hierarchical node selection methods in the training and search stages, respectively, which make better uses of the operation rewards and hierarchical information. Moreover, for a fair comparison of different NAS methods, we construct an open-source NAS benchmark of a macro search space evaluated on CIFAR-10, namely NAS-Bench-Macro. Extensive experiments on NAS-Bench-Macro and ImageNet demonstrate that our method significantly improves search efficiency and performance. For example, by only searching $20$ architectures, our obtained architecture achieves $78.0\%$ top-1 accuracy with 442M FLOPs on ImageNet. Code (Benchmark) is available at: \url{https://github.com/xiusu/NAS-Bench-Macro}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。