We introduce a new function-preserving transformation for efficient neural architecture search. This network transformation allows reusing previously trained networks and existing successful architectures that improves sample efficiency. We aim to address the limitation of current network transformation operations that can only perform layer-level architecture modifications, such as adding (pruning) filters or inserting (removing) a layer, which fails to change the topology of connection paths. Our proposed path-level transformation operations enable the meta-controller to modify the path topology of the given network while keeping the merits of reusing weights, and thus allow efficiently designing effective structures with complex path topologies like Inception models. We further propose a bidirectional tree-structured reinforcement learning meta-controller to explore a simple yet highly expressive tree-structured architecture space that can be viewed as a generalization of multi-branch architectures. We experimented on the image classification datasets with limited computational resources (about 200 GPU-hours), where we observed improved parameter efficiency and better test results (97.70% test accuracy on CIFAR-10 with 14.3M parameters and 74.6% top-1 accuracy on ImageNet in the mobile setting), demonstrating the effectiveness and transferability of our designed architectures.
Hierarchical and Permutation-Invariant Feature Transformation Learning via Policy-Guided Embedding Search
University of Kansas(堪萨斯大学);Southwest University of Finance and Economics(西南财经大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of Auckland(奥克兰大学);Arizona State University(亚利桑那州立大学);Northeast Normal University(东北师范大学)
Collaborative Multi-Agent Reinforcement Learning for Automated Feature Transformation with Graph-Driven Path Optimization
Computer Network Information Center, Chinese Academy of Sciences, and the University of the Chinese Academy of Sciences(中国科学院计算机网络信息中心及中国科学院大学);Department of Electrical Engineering and Computer Science at the University of Kansas(堪萨斯大学电气工程与计算机科学系);School of Computing and Information Technology, Great Bay University, Dongguan, China(广东大湾大学计算机与信息科技学院);School of Software, Tsinghua University(清华大学软件学院);Institute for Infocomm Research, Agency for Science, Technology, and Research(新加坡科技研究局信息与通信研究院)