使用渐进进化搜索神经结构
Neural Architecture Search using Progressive Evolution
摘要:
使用进化算法(EA)的Vanilla神经架构搜索涉及通过从头开始训练来评估每个架构,这非常耗时。这可以通过使用超网来估计搜索空间中的每个架构的适应度来减少,这是由于其权重共享的性质。然而,估计的适应度是非常嘈杂的,由于在超网中的操作的共同适应。在这项工作中,我们提出了一种称为pEvoNAS的方法,其中整个神经架构搜索空间逐渐减少到较小的搜索空间区域具有良好的架构。这是通过在使用遗传算法的架构搜索期间使用用于架构评估的训练超网来找到具有良好架构的搜索空间区域来实现的。在到达最终缩减的搜索空间时,然后使用超网来使用进化在该搜索空间中搜索最佳架构。还通过使用权重继承来增强搜索,其中用于较小搜索空间的超网从用于较大搜索空间的先前训练的超网继承其权重。与先前基于EA的方法相比,pEvoNAS在CIFAR-10和CIFAR-100上给出了更好的结果,同时使用显著更少的计算资源。我们论文的代码可以在https://github.com/nightstorm0909/pEvoNAS上找到
英文摘要:
Vanilla neural architecture search using evolutionary algorithms (EA) involves evaluating each architecture by training it from scratch, which is extremely time-consuming. This can be reduced by using a supernet to estimate the fitness of every architecture in the search space due to its weight sharing nature. However, the estimated fitness is very noisy due to the co-adaptation of the operations in the supernet. In this work, we propose a method called pEvoNAS wherein the whole neural architecture search space is progressively reduced to smaller search space regions with good architectures. This is achieved by using a trained supernet for architecture evaluation during the architecture search using genetic algorithm to find search space regions with good architectures. Upon reaching the final reduced search space, the supernet is then used to search for the best architecture in that search space using evolution. The search is also enhanced by using weight inheritance wherein the supernet for the smaller search space inherits its weights from previous trained supernet for the bigger search space. Exerimentally, pEvoNAS gives better results on CIFAR-10 and CIFAR-100 while using significantly less computational resources as compared to previous EA-based methods. The code for our paper can be found in https://github.com/nightstorm0909/pEvoNAS
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。