随机神经网络和水库系统的逼近界
Approximation Bounds for Random Neural Networks and Reservoir Systems
摘要：
本文研究了基于单隐层前馈和递归神经网络的随机权值逼近问题。这些方法，其中只有最后一层的权重和一些超参数进行了优化，已成功地应用于广泛的静态和动态学习问题。尽管这种方法在经验任务中很受欢迎，但关于未知函数、权重分布和近似率之间关系的重要理论问题仍然没有解决。在这项工作中，它被证明，只要未知的功能，功能，或动力系统是足够的规则，它是可能的随机（递归）神经网络的内部权重从一个通用的分布（不依赖于未知对象）和量化的神经元和超参数的数量方面的错误。特别是，这证明了随机生成的权重的回声状态网络能够任意地近似广泛的一类动力系统，从而为他们在学习动力系统方面的经验观察到的成功提供了第一个数学解释。
英文摘要：
This work studies approximation based on single-hidden-layer feedforward and recurrent neural networks with randomly generated internal weights. These methods, in which only the last layer of weights and a few hyperparameters are optimized, have been successfully applied in a wide range of static and dynamic learning problems. Despite the popularity of this approach in empirical tasks, important theoretical questions regarding the relation between the unknown function, the weight distribution, and the approximation rate have remained open. In this work it is proved that, as long as the unknown function, functional, or dynamical system is sufficiently regular, it is possible to draw the internal weights of the random (recurrent) neural network from a generic distribution (not depending on the unknown object) and quantify the error in terms of the number of neurons and the hyperparameters. In particular, this proves that echo state networks with randomly generated weights are capable of approximating a wide class of dynamical systems arbitrarily well and thus provides the first mathematical explanation for their empirically observed success at learning dynamical systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
