arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2309.06015cs.LGmath.DSmath.OC
收藏

深度神经网络的插值、逼近与可控性

Interpolation, Approximation and Controllability of Deep Neural Networks

National University of Singapore(新加坡国立大学);Peking University(北京大学)

Jingpu Cheng, Qianxiao Li, Ting Lin, Zuowei Shen

AI总结:

通过控制理论研究深度残差神经网络的表示能力,刻画了通用插值与通用逼近的关系及等价条件。

摘要:

我们通过控制理论研究理想化为连续动力系统的深度残差神经网络的表示能力。具体而言,我们考虑源自监督学习的两个属性:通用插值——匹配任意输入和目标训练样本的能力,以及与之密切相关的通用逼近概念——通过流图逼近输入-目标函数关系的能力。在控制族满足仿射不变性的假设下,我们给出了通用插值的刻画,表明它本质上适用于任何具有非线性的架构。此外,我们阐明了通用插值与通用逼近在一般控制系统中的关系,表明这两个属性不能相互推导。同时,我们确定了控制族和目标函数的条件,以确保这两个概念的等价性。

英文摘要:

We investigate the expressive power of deep residual neural networks idealized as continuous dynamical systems through control theory. Specifically, we consider two properties that arise from supervised learning, namely universal interpolation - the ability to match arbitrary input and target training samples - and the closely related notion of universal approximation - the ability to approximate input-target functional relationships via flow maps. Under the assumption of affine invariance of the control family, we give a characterisation of universal interpolation, showing that it holds for essentially any architecture with non-linearity. Furthermore, we elucidate the relationship between universal interpolation and universal approximation in the context of general control systems, showing that the two properties cannot be deduced from each other. At the same time, we identify conditions on the control family and the target function that ensures the equivalence of the two notions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑