Optimal input representation in neural systems at the edge of chaos
Departamento de Electromagnetismo y Física de la Materia(电磁学与物质物理系);Instituto Carlos I de Física Teórica y Computacional(卡洛斯一世理论与计算物理研究所);Universidad de Granada(格拉纳达大学)
Shedding light onto how biological systems represent, process and store information in noisy environments is a key and challenging goal. A stimulating, though controversial, hypothesis poses that operating in dynamical regimes near the edge of a phase transition, i.e. at criticality or the "edge of chaos", can provide information-processing living systems with important operational advantages, creating, e.g., an optimal trade-off between robustness and flexibility. Here, we elaborate on a recent theoretical result, which establishes that the spectrum of covariance matrices of neural networks representing complex inputs in a robust way needs to decay as a power-law of the rank, with an exponent close to unity, a result that has been indeed experimentally verified in neurons of the mouse visual cortex. Aimed at understanding and mimicking these results, we construct an artificial neural network and train it to classify images. Remarkably, we find that the best performance in such a task is obtained when the network operates near the critical point, at which the eigenspectrum of the covariance matrix follows the very same statistics as actual neurons do. Thus, we conclude that operating near criticality can also have -- besides the usually alleged virtues -- the advantage of allowing for flexible, robust and efficient input representations.
Input correlations impede suppression of chaos and learning in balanced rate networks
Zuckerman Mind, Brain, Behavior Institute, Columbia University(哥伦比亚大学祖克曼心智、大脑与行为研究所);The Abdus Salam International Centre for Theoretical Physics(阿卜杜勒·萨拉姆国际理论物理中心);Institute of Genetics, University of Bonn(波恩大学遗传学研究所)
Institut für theoretische Physik(理论物理研究所);Chung-Ang University(中央大学);Centro Brasileiro de Pesquisas Físicas(巴西物理研究中心);University of Geneva(日内瓦大学);Stony Brook University(石溪大学)
The edge of chaos: quantum field theory and deep neural networks
Max-Planck-Institut für Physik komplexer Systeme(马克斯·普朗克复杂系统物理研究所);Würzburg-Dresden Cluster of Excellence ct.qmat(维尔茨堡-德累斯顿卓越集群ct.qmat);Instituut-Lorentz, Universiteit Leiden(莱顿大学洛伦兹研究所);Nordita KTH Royal Institute of Technology and Stockholm University(瑞典皇家理工学院和斯德哥尔摩大学北欧理论物理研究所)
Learning Chaotic Dynamics with Neuromorphic Network Dynamics
School of Physics, University of Sydney, Sydney, NSW 2006, Australia.(悉尼大学物理系);School of Mathematics and Statistics, University of Sydney, Sydney, NSW 2006, Australia(悉尼大学数学与统计学系);Centre for Complex Systems, University of Sydney, Sydney, NSW 2006, Australia.(悉尼大学复杂系统中心)