Echo state networks are a recently developed type of recurrent neural network where the internal layer is fixed with random weights, and only the output layer is trained on specific data. Echo state networks are increasingly being used to process spatiotemporal data in real-world settings, including speech recognition, event detection, and robot control. A strength of echo state networks is the simple method used to train the output layer - typically a collection of linear readout weights found using a least squares approach. Although straightforward to train and having a low computational cost to use, this method may not yield acceptable accuracy performance on noisy data.
This study compares the performance of three echo state network output layer methods to perform classification on noisy data: using trained linear weights, using sparse trained linear weights, and using trained low-rank approximations of reservoir states. The methods are investigated experimentally on both synthetic and natural datasets. The experiments suggest that using regularized least squares to train linear output weights is superior on data with low noise, but using the low-rank approximations may significantly improve accuracy on datasets contaminated with higher noise levels.
A characterization of the Edge of Criticality in Binary Echo State Networks
回声状态网络（ESN）是一种简化的递归神经网络模型，由一个库和一个线性的、可训练的读出层组成。水库是可调的一些超参数，控制网络的行为。已知ESN在被配置在称为\emph{Edge of Criticality}（EoC）的（超）参数空间中的区域上时在解决任务方面是有效的，其中系统对扰动最敏感，从而影响其行为。在本文中，我们提出了二进制ESN，这是架构上等同于标准ESN，但考虑二进制激活函数和二进制经常性的权重。对于这些网络，我们推导出一个封闭形式的表达EoC在自治的情况下，并进行模拟，以评估其行为的情况下，嘈杂的神经元和存在的信号。我们提出了一个理论解释的事实，即方差的输入起着重要的作用，在表征的EoC。
Evolutionary Echo State Network: evolving reservoirs in the Fourier space
Faculty of Electrical Engineering and Computer Science（电气工程与计算机科学学院）；VŠB-Technical University of Ostrava（俄斯特拉发技术大学）；INRIA Rennes – Bretagne Atlantique（法国国家信息与自动化研究所雷恩-布列塔尼大西洋中心）
Improving the Performance of Echo State Networks Through State Feedback
Wyant College of Optical Sciences, University of Arizona（亚利桑那大学怀恩特光学科学学院）；School of Electrical Engineering and Telecommunications, University of New South Wales（新南威尔士大学电气工程与电信学院）
