arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2210.02373cs.LGcs.NAmath.DSmath.NA
收藏

基于动力系统的神经网络

Dynamical systems' based neural networks

NTNU(挪威科技大学);University of Cambridge(剑桥大学)

Elena Celledoni, Davide Murari, Brynjulf Owren, Carola-Bibiane Schönlieb, Ferdia Sherry

AI总结:

本文提出基于非自治常微分方程和保持结构的数值离散化构建神经网络,注入几何结构并提升理论理解,给出通用逼近结果,并展示1-Lipschitz架构在CIFAR数据集上的表达力与对抗鲁棒性。

摘要:

神经网络因其在许多应用中的有效性而引起了广泛关注。然而,它们的数学性质通常尚未被充分理解。如果数据或待逼近函数存在某种内在的几何结构,那么在神经网络的设计中考虑这一结构通常是可取的。在本工作中,我们从非自治常微分方程(ODE)出发,利用合适的、保持结构的数值时间离散化方法构建神经网络。神经网络的结构随后从ODE向量场的性质中推断出来。除了向网络架构中注入更多结构外,这种建模过程还使得对其行为的理论理解更加深入。我们给出了两个通用逼近结果,并展示了如何在神经网络上施加某些特定性质。特别关注的是1-Lipschitz架构,其中包括非1-Lipschitz的层。这些网络具有表达力,并且对对抗攻击具有鲁棒性,如CIFAR-10和CIFAR-100数据集所示。

英文摘要:

Neural networks have gained much interest because of their effectiveness in many applications. However, their mathematical properties are generally not well understood. If there is some underlying geometric structure inherent to the data or to the function to approximate, it is often desirable to take this into account in the design of the neural network. In this work, we start with a non-autonomous ODE and build neural networks using a suitable, structure-preserving, numerical time-discretisation. The structure of the neural network is then inferred from the properties of the ODE vector field. Besides injecting more structure into the network architectures, this modelling procedure allows a better theoretical understanding of their behaviour. We present two universal approximation results and demonstrate how to impose some particular properties on the neural networks. A particular focus is on 1-Lipschitz architectures including layers that are not 1-Lipschitz. These networks are expressive and robust against adversarial attacks, as shown for the CIFAR-10 and CIFAR-100 datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑