arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2107.05709cond-mat.dis-nncs.LGcs.NEq-bio.NC
收藏

混沌边缘神经系统的最优输入表示

Optimal input representation in neural systems at the edge of chaos

Departamento de Electromagnetismo y Física de la Materia(电磁学与物质物理系);Instituto Carlos I de Física Teórica y Computacional(卡洛斯一世理论与计算物理研究所);Universidad de Granada(格拉纳达大学)

Guillermo B. Morales, Miguel A. Muñoz

摘要:

阐明生物系统如何在嘈杂的环境中表示,处理和存储信息是一个关键和具有挑战性的目标。一个令人兴奋的,尽管有争议的假设提出,在相变边缘附近的动力学机制中操作,即在临界或“混沌边缘”,可以为信息处理生命系统提供重要的操作优势,例如,鲁棒性和灵活性之间的最佳折衷。在这里,我们阐述了最近的理论结果,它建立了一个强大的方式表示复杂的输入的神经网络的协方差矩阵的频谱需要衰减的秩的幂律,与指数接近统一,一个结果,确实已经实验验证了在小鼠视觉皮层的神经元。为了理解和模仿这些结果,我们构建了一个人工神经网络,并训练它来分类图像。值得注意的是,我们发现当网络在临界点附近运行时,在这样的任务中获得最佳性能,在临界点处,协方差矩阵的特征谱与实际神经元的统计数据相同。因此,我们的结论是,近临界操作也可以有-除了通常声称的优点-允许灵活,强大和有效的输入表示的优势。

英文摘要:

Shedding light onto how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e. at criticality or the "edge of chaos", can provide information-processing living systems with important operational advantages, creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent theoretical result, which establishes that the spectrum of covariance matrices of neural networks representing complex inputs in a robust way needs to decay as a power-law of the rank, with an exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural network and train it to classify images. Remarkably, we find that the best performance in such a task is obtained when the network operates near the critical point, at which the eigenspectrum of the covariance matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near criticality can also have -- besides the usually alleged virtues -- the advantage of allowing for flexible, robust and efficient input representations.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑