arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.07266cs.NE
收藏

基于协方差矩阵自适应演化策略的神经网络结构搜索

Neural Architecture Search using Covariance Matrix Adaptation Evolution Strategy

Nilotpal Sinha, Kuan-Wen Chen

摘要:

基于进化的神经结构搜索需要大量的计算资源,导致搜索时间较长。在这项工作中,我们提出了一个框架,应用协方差矩阵自适应演化策略(CMA-ES)的神经结构搜索问题,称为CMANAS,取得了更好的结果比以前的基于进化的方法,同时减少了搜索时间显着。架构建模使用正态分布,这是更新使用CMA-ES的基础上采样人口的健身。我们在验证数据上使用训练的单次模型(OSM)的准确性作为对单个架构的适应度的预测,以减少搜索时间。我们还使用了一个架构适应度表(AF表),用于记录已经评估的架构,从而进一步减少搜索时间。CMANAS在CIFAR-10上完成了架构搜索,0.45 GPU天的前1测试准确率为97.44%,在CIFAR-100上完成了单GPU 0.6 GPU天的前1测试准确率为83.24%。然后将CIFAR-10和CIFAR-100上搜索的顶级架构转移到ImageNet,分别达到92.6%和92.1%的前5名准确率。

英文摘要:

Evolution-based neural architecture search requires high computational resources, resulting in long search time. In this work, we propose a framework of applying the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) to the neural architecture search problem called CMANAS, which achieves better results than previous evolution-based methods while reducing the search time significantly. The architectures are modelled using a normal distribution, which is updated using CMA-ES based on the fitness of the sampled population. We used the accuracy of a trained one shot model (OSM) on the validation data as a prediction of the fitness of an individual architecture to reduce the search time. We also used an architecture-fitness table (AF table) for keeping record of the already evaluated architecture, thus further reducing the search time. CMANAS finished the architecture search on CIFAR-10 with the top-1 test accuracy of 97.44% in 0.45 GPU day and on CIFAR-100 with the top-1 test accuracy of 83.24% for 0.6 GPU day on a single GPU. The top architectures from the searches on CIFAR-10 and CIFAR-100 were then transferred to ImageNet, achieving the top-5 accuracy of 92.6% and 92.1%, respectively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑