arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.07173cs.NE
收藏

基于笛卡尔遗传编程编码方法的神经结构搜索

Neural Architecture Search based on Cartesian Genetic Programming Coding Method

Xuan Wu, Linhan Jia, Xiuyi Zhang, Liang Chen, Yanchun Liang, You Zhou, Chunguo Wu

摘要:

神经架构搜索(NAS)是自动化机器学习领域的一个热门话题,在相当多的机器学习任务上,它在设计神经架构方面优于人类。受笛卡尔遗传编程(CGP)神经网络自然表示形式的启发,提出了一种基于CGP的进化NAS方法(CGPNAS)来解决句子分类问题。为了在CGP框架下对体系结构进行进化,将卷积等运算识别为CGP的功能节点类型,并基于进化策略设计了进化运算。实验结果表明,搜索到的架构与人类设计的架构的性能相当。我们验证了我们的进化架构的域转移的能力。传递实验结果表明,精度损失小于2- 5%。最后,消融研究将注意力功能确定为单个关键功能节点,并且沿着线性变换可以保持与完全进化的架构相似的精度,这值得在未来进行研究。

英文摘要:

Neural architecture search (NAS) is a hot topic in the field of automated machine learning and outperforms humans in designing neural architectures on quite a few machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. To evolve the architectures under the framework of CGP, the operations such as convolution are identified as the types of function nodes of CGP, and the evolutionary operations are designed based on Evolutionary Strategy. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures. We verify the ability of domain transfer of our evolved architectures. The transfer experimental results show that the accuracy deterioration is lower than 2-5%. Finally, the ablation study identifies the Attention function as the single key function node and the linear transformations along could keep the accuracy similar with the full evolved architectures, which is worthy of investigation in the future.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑