反馈神经网络中内部表征的质量影响学习性能
Quality of internal representation shapes learning performance in feedback neural networks
摘要:
复杂生物系统的一个基本特征是与环境形成反馈相互作用的能力。研究这种相互作用的一个突出模型是水库计算,其中学习作用于低维瓶颈。尽管这种学习方案很简单,但对水库网络训练成功的影响因素或阻碍因素通常还没有得到很好的理解。在这项工作中,我们研究了训练生成正弦信号的非线性反馈网络,并分析了内部网络动态和目标属性之间的相互作用如何塑造学习性能。通过对线性化网络进行精确的数学分析,我们预测,当目标的特征在于最佳中频时,学习性能最大化,该中频随着内部水库连通性的强度单调下降。在最佳频率处,目标信号的储层表示是高维的、去同步的,并且因此对噪声最大鲁棒。我们表明,我们的预测成功地捕捉到了非线性网络性能的定性行为。此外,我们发现内部表征和性能之间的关系可以在训练的非线性网络中进一步利用,以解释没有线性对应物的行为。我们的研究结果表明,学习成功的一个主要决定因素是目标的内部表征的质量,这反过来又是由控制内部网络和那些定义任务的参数之间的相互作用形成的。
英文摘要:
A fundamental feature of complex biological systems is the ability to form feedback interactions with their environment. A prominent model for studying such interactions is reservoir computing, where learning acts on low-dimensional bottlenecks. Despite the simplicity of this learning scheme, the factors contributing to or hindering the success of training in reservoir networks are in general not well understood. In this work, we study non-linear feedback networks trained to generate a sinusoidal signal, and analyze how learning performance is shaped by the interplay between internal network dynamics and target properties. By performing exact mathematical analysis of linearized networks, we predict that learning performance is maximized when the target is characterized by an optimal, intermediate frequency which monotonically decreases with the strength of the internal reservoir connectivity. At the optimal frequency, the reservoir representation of the target signal is high-dimensional, de-synchronized, and thus maximally robust to noise. We show that our predictions successfully capture the qualitative behaviour of performance in non-linear networks. Moreover, we find that the relationship between internal representations and performance can be further exploited in trained non-linear networks to explain behaviours which do not have a linear counterpart. Our results indicate that a major determinant of learning success is the quality of the internal representation of the target, which in turn is shaped by an interplay between parameters controlling the internal network and those defining the task.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。