液态机（Liquid State Machine，LSM）又称脉冲神经网络（Spiking Neural Networks，SNN）的递归版本，由于其计算能力强、大脑的生物相容性、结构简单、训练复杂度低等优点，引起了人们的极大兴趣。通过探索网络结构和参数的设计空间，最近的工作已经证明了在低复杂度下提高LSM模型精度的巨大潜力。然而，这些工作都是基于手动定义的网络架构或预定义的参数。考虑到大脑结构的多样性和独特性，LSM模型的设计应在尽可能大的搜索空间中进行探索。在本文中，我们提出了一个神经架构搜索（NAS）为基础的框架，以探索自动面向队列的LSM模型的架构和参数设计空间。为了处理呈指数级增长的设计空间，我们采用了三步搜索LSM，包括多液体结构搜索，神经元数量的变化和参数搜索，如每个液体内的连通性百分比和兴奋性神经元比例。此外，我们提出使用模拟退火（SA）算法来实现三步启发式搜索。三个数据集，包括MNIST和NMNIST的图像数据集和FSDD的语音数据集，被用来测试我们提出的框架的有效性。仿真结果表明，我们提出的框架可以产生面向小卫星的最优LSM模型，具有高精度和低复杂度。在仅使用1000个尖峰神经元的情况下，在三个数据集上的最佳分类准确率分别为93.2%、92.5%和84%，与单个LSM相比，网络连接数平均减少了61.4%。此外，我们发现，在三个数据集上的最优LSM模型中的神经元总数可以进一步减少20%，而精度损失仅约0.5%。
英文摘要：
Liquid State Machine (LSM), also known as the recurrent version of Spiking Neural Networks (SNN), has attracted great research interests thanks to its high computational power, biological plausibility from the brain, simple structure and low training complexity. By exploring the design space in network architectures and parameters, recent works have demonstrated great potential for improving the accuracy of LSM model with low complexity. However, these works are based on manually-defined network architectures or predefined parameters. Considering the diversity and uniqueness of brain structure, the design of LSM model should be explored in the largest search space possible. In this paper, we propose a Neural Architecture Search (NAS) based framework to explore both architecture and parameter design space for automatic dataset-oriented LSM model. To handle the exponentially-increased design space, we adopt a three-step search for LSM, including multi-liquid architecture search, variation on the number of neurons and parameters search such as percentage connectivity and excitatory neuron ratio within each liquid. Besides, we propose to use Simulated Annealing (SA) algorithm to implement the three-step heuristic search. Three datasets, including image dataset of MNIST and NMNIST and speech dataset of FSDD, are used to test the effectiveness of our proposed framework. Simulation results show that our proposed framework can produce the dataset-oriented optimal LSM models with high accuracy and low complexity. The best classification accuracy on the three datasets is 93.2%, 92.5% and 84% respectively with only 1000 spiking neurons, and the network connections can be averagely reduced by 61.4% compared with a single LSM. Moreover, we find that the total quantity of neurons in optimal LSM models on three datasets can be further reduced by 20% with only about 0.5% accuracy loss.
A Neural Architecture Search Method using Auxiliary Evaluation Metric based on ResNet Architecture
Key Laboratory of Intelligent Computing；Information Processing, Ministry of Education, School of Computer Science \& School of Cyberspace Security, Xiangtan University Xiangtan Hunan China；Information Processing, Ministry of Education, School of Computer Science \& School of Cyberspace Security, Xiangtan University
