在这项工作中,我们考虑了一个深度神经学习的模型问题,即假设我们可以访问给定函数在有限点集上的点值时,对该给定函数进行学习。深度神经网络插值器是 f 的最终近似,它是通过涉及给定 DNN 架构和优化步骤的典型机器学习算法获得的,并假设该优化步骤被精确求解。这些是基于神经网络的最简单的回归算法之一。在这项工作中,我们引入了一种估计(泛化)误差和收敛性的新方法。我们的结果包括:(i) 在不对神经网络作任何结构假设,且仅对学习函数 f 施加温和的正则性假设的情况下,给出误差估计;(ii) 仅要求神经网络空间具备适当的逼近能力,即可实现近似对目标函数 f 的收敛。
英文摘要:
In this work we consider a model problem of deep neural learning, namely the learning of a given function when it is assumed that we have access to its point values on a finite set of points. The deep neural network interpolant is the the resulting approximation of f, which is obtained by a typical machine learning algorithm involving a given DNN architecture and an optimisation step, which is assumed to be solved exactly. These are among the simplest regression algorithms based on neural networks. In this work we introduce a new approach to the estimation of the (generalisation) error and to convergence. Our results include (i) estimates of the error without any structural assumption on the neural networks and under mild regularity assumptions on the learning function f (ii) convergence of the approximations to the target function f by only requiring that the neural network spaces have appropriate approximation capability.
Memorisation, convergence and generalisation in generative models
INRIA Paris & DI ENS, PSL University, Paris, France(巴黎国家信息与自动化研究所(INRIA)及巴黎高等师范学院(ENS)与巴黎大学(PSL University));International School of Advanced Studies (SISSA), Trieste, Italy(意大利国际高级研究学院(SISSA))
A PAC-Bayesian View of Generalisation for Physics-Informed Machine Learning
Université Jean Monnet Saint-Étienne, CNRS, Institut d’Optique Graduate School, Laboratoire Hubert Curien UMR 5516(里昂蒙特大学圣埃蒂安分校、法国国家科学研究中心、光学研究生院、Hubert Curien实验室 UMR 5516);Inria and University College London, France and United Kingdom(Inria 和 英国伦敦大学学院,法国和英国)
University of Tübingen(蒂宾根大学);Centre for Integrative Neuroscience, University of Tübingen(蒂宾根大学整合神经科学中心);International Max Planck Research School for Intelligent Systems(国际马克斯·普朗克智能系统研究院);Graduate School of Neural and Behavioural Sciences, University of Tübingen(蒂宾根大学神经与行为科学研究生院);University of Potsdam(波茨坦大学);Bernstein Center for Computational Neuroscience Tübingen(蒂宾根伯恩斯坦计算神经科学中心);Max Planck Institute for Biological Cybernetics(马克斯·普朗克生物控制论研究所);Max Planck Institute for Intelligent Systems(马克斯·普朗克智能系统研究所)