脉冲神经网络中的内部表征、临界性与重整化群
Internal Representations in Spiking Neural Networks, criticality and the Renormalization Group
AI总结:
本文研究深度脉冲神经网络内部表征的涨落特征,发现宽尾分布与持续关联并非源于临界相变,而是类似铁磁系统中磁场反转驱动的涨落,并从重整化群视角进行解释。
摘要:
在若干例子中,外周感觉系统的最优信息处理已被关联到临界或近临界状态的特征。此外,在清醒和麻醉的实验模型中,无论是在体外还是在体内,皮层系统也都被描述为处于临界状态。我们研究类似的特征是否刻画了一个多层(深度)脉冲人工神经网络的内部表征(IR),该网络执行计算上简单但有意义的认知任务,所采用的方法论受生物学设置启发,即在大鼠皮层植入电极,大鼠或自由活动或处于不同麻醉水平。当网络输入改变时,所发现的内部表征涨落关联衰减特征时间的增加,表明内部表征涨落具有宽尾分布。即使网络尚不能执行分类任务——无论是由于部分训练,还是由于简单模型中低剂量麻醉的影响——宽尾仍然存在。然而,我们没有找到足够的证据表明雪崩规模和持续时间服从幂律分布。我们从重整化群的角度解释这些结果,指出尽管存在宽尾,但这与临界相变无关,而更类似于铁磁系统中由磁场反转驱动的涨落。还构造了另一个非临界系统涨落持续关联的例子,其中粒子在缓慢变化的势场上做布朗运动。
英文摘要:
Optimal information processing in peripheral sensory systems has been associated in several examples to the signature of a critical or near critical state. Furthermore, cortical systems have also been described to be in a critical state in both wake and anesthetized experimental models, both {\it in vitro} and {\it in vivo}. We investigate whether a similar signature characterizes the internal representations (IR) of a multilayer (deep) spiking artificial neural network performing computationally simple but meaningful cognitive tasks, using a methodology inspired in the biological setup, with cortical implanted electrodes in rats, either freely behaving or under different levels of anesthesia. The increase of the characteristic time of the decay of the correlation of fluctuations of the IR, found when the network input changes, are indications of a broad-tailed distribution of IR fluctuations. The broad tails are present even when the network is not yet capable of performing the classification tasks, either due to partial training or to the effect of a low dose of anesthesia in a simple model. However, we don't find enough evidence of power law distributions of avalanche size and duration. We interpret the results from a renormalization group perspective to point out that despite having broad tails, this is not related to a critical transition but rather similar to fluctuations driven by the reversal of the magnetic field in a ferromagnetic system. Another example of persistent correlation of fluctuations of a non critical system is constructed, where a particle undergoes Brownian motion on a slowly varying potential.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。