优化学习的人工神经网络裁剪
Tailoring Artificial Neural Networks for Optimal Learning
摘要：
作为递归神经网络最重要的范例之一，回声状态网络（ESN）已被广泛应用于从机器人到医学、金融和语言处理等领域。ESN范式的一个关键特征是它的水库-一个有向和加权的神经元网络，它将输入的时间序列投影到一个可以应用线性回归或分类的高维空间。尽管进行了广泛的研究，水库网络对ESN性能的影响仍然不清楚。结合物理学、动力系统和网络科学的工具，我们试图打开ESN的黑箱，并为理解一般人工神经网络的行为提供见解。通过水库网络的频谱分析，我们揭示了一个关键因素，在很大程度上决定了ESN的内存容量，从而影响其性能。此外，我们发现，在水库网络中添加短循环可以为特定任务定制ESN并优化学习。我们验证了我们的研究结果，应用ESN预测合成和真实的基准时间序列。我们的研究结果提供了一种新的方法来设计特定任务的ESN。更重要的是，它展示了将物理学、动力系统和网络科学的工具相结合的力量，为理解一般人工神经网络的机制提供了新的见解。
英文摘要：
As one of the most important paradigms of recurrent neural networks, the echo state network (ESN) has been applied to a wide range of fields, from robotics to medicine, finance, and language processing. A key feature of the ESN paradigm is its reservoir --- a directed and weighted network of neurons that projects the input time series into a high dimensional space where linear regression or classification can be applied. Despite extensive studies, the impact of the reservoir network on the ESN performance remains unclear. Combining tools from physics, dynamical systems and network science, we attempt to open the black box of ESN and offer insights to understand the behavior of general artificial neural networks. Through spectral analysis of the reservoir network we reveal a key factor that largely determines the ESN memory capacity and hence affects its performance. Moreover, we find that adding short loops to the reservoir network can tailor ESN for specific tasks and optimize learning. We validate our findings by applying ESN to forecast both synthetic and real benchmark time series. Our results provide a new way to design task-specific ESN. More importantly, it demonstrates the power of combining tools from physics, dynamical systems and network science to offer new insights in understanding the mechanisms of general artificial neural networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
