arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1909.05176cs.LGcs.NEnlin.AOnlin.CDstat.ML
收藏

混沌边缘的最优机器智能

Optimal Machine Intelligence at the Edge of Chaos

Institute of High Performance Computing, A*STAR(新加坡科技研究局高性能计算研究所);National University of Singapore(新加坡国立大学);Centre for Quantum Technologies, National University of Singapore(新加坡国立大学量子技术中心);NUS Graduate School for Integrative Sciences and Engineering(新加坡国立大学综合科学与工程研究生院)

Ling Feng, Lin Zhang, Choy Heng Lai

摘要:

长期以来,人们一直认为,生物大脑在两个不同阶段之间的某个临界点上运行,可能是秩序和混乱。尽管有许多来自大脑的间接经验证据和对简单神经网络的分析指示,但这一关于通用非线性系统的假设的基础仍然不清楚。在这里,我们开发了一个通用的理论,揭示了确切的边缘的混沌之间的边界的混沌阶段和(伪)周期阶段所产生的Neimark-Sacker分岔。该边由非线性算子的渐近雅可比范数值解析确定,并受系统维数的影响。混沌边缘的最优性与输入和输出之间的最高信息传递有关,类似于Logistic映射。作为经验验证,我们对计算机视觉中各种深度学习模型的实验证明了模型在混沌边缘附近的最优性,我们观察到最先进的训练算法将模型推向这种边缘,因为它们变得更加准确。我们通过渐近稳定性进一步建立了对深度学习模型泛化的理论理解。

英文摘要:

It has long been suggested that the biological brain operates at some critical point between two different phases, possibly order and chaos. Despite many indirect empirical evidence from the brain and analytical indication on simple neural networks, the foundation of this hypothesis on generic non-linear systems remains unclear. Here we develop a general theory that reveals the exact edge of chaos is the boundary between the chaotic phase and the (pseudo)periodic phase arising from Neimark-Sacker bifurcation. This edge is analytically determined by the asymptotic Jacobian norm values of the non-linear operator and influenced by the dimensionality of the system. The optimality at the edge of chaos is associated with the highest information transfer between input and output at this point similar to that of the logistic map. As empirical validations, our experiments on the various deep learning models in computer vision demonstrate the optimality of the models near the edge of chaos, and we observe that the state-of-art training algorithms push the models towards such edge as they become more accurate. We further establishes the theoretical understanding of deep learning model generalization through asymptotic stability.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑