Neural architecture search (NAS) is a hot topic in the field of automated machine learning and outperforms humans in designing neural architectures on quite a few machine learning tasks. Motivated by the natural representation form of neural networks by the Cartesian genetic programming (CGP), we propose an evolutionary approach of NAS based on CGP, called CGPNAS, to solve sentence classification task. To evolve the architectures under the framework of CGP, the operations such as convolution are identified as the types of function nodes of CGP, and the evolutionary operations are designed based on Evolutionary Strategy. The experimental results show that the searched architectures are comparable with the performance of human-designed architectures. We verify the ability of domain transfer of our evolved architectures. The transfer experimental results show that the accuracy deterioration is lower than 2-5%. Finally, the ablation study identifies the Attention function as the single key function node and the linear transformations along could keep the accuracy similar with the full evolved architectures, which is worthy of investigation in the future.
Cartesian Genetic Programming Approach for Designing Convolutional Neural Networks
AGH University of Krakow(克拉科夫AGH大学);Systems Research Institute, Polish Academy of Sciences(波兰科学院系统研究所);University of Technology Sydney(悉尼科技大学);Óbuda University(奥布达大学)
Node Preservation and its Effect on Crossover in Cartesian Genetic Programming
Department of Computer Science(计算机科学系);BEACON Center for the Study of Evolution in Action Michigan State University, East Lansing, MI 48824, USA(进化机制行动研究中心 密歇根州立大学 埃文斯兰斯)
Facultad de Ciencias, Universidad Nacional Autónoma de México(墨西哥国立自治大学理学院);Centro de Investigación y Estudios Avanzados(高级研究与调查中心);International Max Planck Research School for Computational Biology and Scientific Computing(国际马克斯·普朗克计算生物学与科学计算研究学院);Freie Universität Berlin(柏林自由大学);Centro de Ciencias de la Complejidad, Universidad Nacional Autónoma de México(墨西哥国立自治大学复杂性科学中心)
Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Center for Long-term Artificial Intelligence(长期人工智能中心);Key Laboratory of Brain Cognition and Brain-inspired Intelligence Technology, Chinese Academy of Sciences(中国科学院脑认知与脑智能技术重点实验室)