arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.05959cs.LGcond-mat.dis-nncs.AIcs.NE
收藏

自然启发的局部传播

Nature-Inspired Local Propagation

Inria(法国国家信息与自动化研究所);University of Siena(锡耶纳大学)

Alessandro Betti, Marco Gori

AI总结:

本文提出一种受自然启发的时空局部学习算法,其哈密顿方程形式在传播速度无穷时退化为反向传播,为在线机器学习提供新途径。

摘要:

机器学习领域取得的惊人成果,包括生成式AI的最新进展,依赖于大规模数据集。相反,自然界中的智能过程无需此类数据集,仅通过对环境信息的在线处理即可产生。特别是,自然学习过程依赖于数据表示与学习相互交织、尊重时空局部性的机制。本文表明,这一特性源于一种受理论物理学相关研究启发的算法前学习视角。我们证明,所推导的“学习定律”(其结构采用哈密顿方程形式)的算法解释,在传播速度趋于无穷时退化为反向传播。这为基于完全在线信息处理的机器学习研究打开了大门,该研究以所提出的时空局部算法替代反向传播。

英文摘要:

The spectacular results achieved in machine learning, including the recent advances in generative AI, rely on large data collections. On the opposite, intelligent processes in nature arises without the need for such collections, but simply by online processing of the environmental information. In particular, natural learning processes rely on mechanisms where data representation and learning are intertwined in such a way to respect spatiotemporal locality. This paper shows that such a feature arises from a pre-algorithmic view of learning that is inspired by related studies in Theoretical Physics. We show that the algorithmic interpretation of the derived "laws of learning", which takes the structure of Hamiltonian equations, reduces to Backpropagation when the speed of propagation goes to infinity. This opens the doors to machine learning studies based on full on-line information processing that are based the replacement of Backpropagation with the proposed spatiotemporal local algorithm.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑