arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2403.07035cs.NEcs.LG
收藏

多种群交替进化神经架构搜索

Multiple Population Alternate Evolution Neural Architecture Search

Xiangtan University(湘潭大学)

Juan Zou, Han Chu, Yizhang Xia, Junwen Xu, Yuan Liu, Zhanglu Hou

AI总结:

针对现有进化神经架构搜索方法在搜索成本与结构多样性间难以兼顾的问题,提出多种群交替进化搜索范式MPAE,通过分单元循环搜索与种群迁移机制,以极低搜索成本在CIFAR数据集上取得最优结果。

摘要:

进化神经架构搜索(Evolutionary Neural Architecture Search, ENAS)的有效性受搜索空间设计的影响。然而,全局搜索空间、可扩展搜索空间、分层搜索空间等常用方法均存在一定局限性。具体而言,全局搜索空间需要大量计算资源与时间,可扩展搜索空间牺牲了网络结构的多样性,分层搜索空间则以增加搜索成本为代价换取网络多样性。\n为解决上述局限,我们提出一种新型神经网络架构搜索范式,并设计了多种群交替进化神经架构搜索(Multiple Population Alternate Evolution Neural Architecture Search, MPAE),该方法能以更低的搜索成本实现模块多样性。MPAE将搜索空间转化为L个相互连接的单元并按顺序搜索这些单元,随后对整个网络的上述搜索过程循环多次,以降低前期单元对后续单元的影响。为加速种群进化过程,我们还提出了种群迁移机制,该机制建立优质迁移档案,并将迁移档案中的优质知识与经验传递给新种群。\n所提方法仅需0.3 GPU天即可在CIFAR数据集上搜索出一个神经网络,且取得了当前最优(state-of-the-art)结果。

英文摘要:

The effectiveness of Evolutionary Neural Architecture Search (ENAS) is influenced by the design of the search space. Nevertheless, common methods including the global search space, scalable search space and hierarchical search space have certain limitations. Specifically, the global search space requires a significant amount of computational resources and time, the scalable search space sacrifices the diversity of network structures and the hierarchical search space increases the search cost in exchange for network diversity. To address above limitation, we propose a novel paradigm of searching neural network architectures and design the Multiple Population Alternate Evolution Neural Architecture Search (MPAE), which can achieve module diversity with a smaller search cost. MPAE converts the search space into L interconnected units and sequentially searches the units, then the above search of the entire network be cycled several times to reduce the impact of previous units on subsequent units. To accelerate the population evolution process, we also propose the the population migration mechanism establishes an excellent migration archive and transfers the excellent knowledge and experience in the migration archive to new populations. The proposed method requires only 0.3 GPU days to search a neural network on the CIFAR dataset and achieves the state-of-the-art results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑