arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.08451math.NAcs.LGcs.NA
收藏

Kolmogorov-Arnold 证明可能启发神经网络学习

The Proof of Kolmogorov-Arnold May Illuminate Neural Network Learning

Harvard University(哈佛大学)

Michael H. Freedman

AI总结:

本文基于 Kolmogorov-Arnold 证明,将多元函数表示分为两步层间映射,解释其模式为 Jacobian 矩阵的“微集中”与高阶稀疏性,论证了该稀疏性如何促成深度神经网络中高阶概念的出现,并提出了两类验证实验。

摘要:

Kolmogorov 和 Arnold 在回答 Hilbert 第 13 问题(在连续函数语境下)时,为现代 Neural Networks (NNs) 理论奠定了基础。他们的证明将多元函数的表示分为两步:第一步(非线性)层间映射给出了数据流形到单个隐藏层的通用嵌入,其图像以特定方式形成模式,从而可定义随后的动态来求解第二步层间映射。我将这种模式解释为几乎处处有定义的层间映射 Jacobian 矩阵的“微集中”。微集中等价于 Jacobian 高阶外幂的稀疏性。我们提出了一个概念性论证,说明这种稀疏性如何为当今深度 NNs 中连续高阶概念的出现奠定基础,并建议了两类实验来检验该假设。

英文摘要:

Kolmogorov and Arnold, in answering Hilbert's 13th problem (in the context of continuous functions), laid the foundations for the modern theory of Neural Networks (NNs). Their proof divides the representation of a multivariate function into two steps: The first (non-linear) inter-layer map gives a universal embedding of the data manifold into a single hidden layer whose image is patterned in such a way that a subsequent dynamic can then be defined to solve for the second inter-layer map. I interpret this pattern as "minor concentration" of the almost everywhere defined Jacobians of the interlayer map. Minor concentration amounts to sparsity for higher exterior powers of the Jacobians. We present a conceptual argument for how such sparsity may set the stage for the emergence of successively higher order concepts in today's deep NNs and suggest two classes of experiments to test this hypothesis.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑