深度流形第一部分:神经网络流形的解剖学
Deep Manifold Part 1: Anatomy of Neural Network Manifold
AI总结:
本文基于数值流形方法原理提出了神经网络流形的数学框架Deep Manifold,揭示了神经网络的五大特性,定义了学习空间与内在路径等概念,并提出了关于训练完成定义、收敛点及token时间戳重要性的三个基本问题。
摘要:
基于数值流形方法原理,我们开发了神经网络流形的数学框架:Deep Manifold,并发现神经网络:1)是结合正向与逆向的数值计算;2)具有近乎无限的自由度;3)学习容量随深度呈指数增长;4)具有自推进的边界条件;5)存在训练隐藏瓶颈。我们还定义了两个概念:神经网络学习空间和深度流形空间,并引入了两个概念:神经网络内在路径和不动点。我们提出了三个基本问题:1)训练完成的定义是什么;2)深度学习的收敛点(神经网络不动点)在哪里;3)鉴于负时间在逆问题中至关重要,训练数据中的token时间戳有多重要。
英文摘要:
Based on the numerical manifold method principle, we developed a mathematical framework of a neural network manifold: Deep Manifold and discovered that neural networks: 1) is numerical computation combining forward and inverse; 2) have near infinite degrees of freedom; 3) exponential learning capacity with depth; 4) have self-progressing boundary conditions; 5) has training hidden bottleneck. We also define two concepts: neural network learning space and deep manifold space and introduce two concepts: neural network intrinsic pathway and fixed point. We raise three fundamental questions: 1). What is the training completion definition; 2). where is the deep learning convergence point (neural network fixed point); 3). How important is token timestamp in training data given negative time is critical in inverse problem.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。