Motivated by conforming finite element methods for elliptic problems of second order, we analyze the approximation of the gradient of a target function by continuous piecewise polynomial functions over a simplicial mesh. The main result is that the global best approximation error is equivalent to an appropriate sum in terms of the local best approximations errors on elements. Thus, requiring continuity does not downgrade local approximability and discontinuous piecewise polynomials essentially do not offer additional approximation power, even for a fixed mesh. This result implies error bounds in terms of piecewise regularity over the whole admissible smoothness range. Moreover, it allows for simple local error functionals in adaptive tree approximation of gradients.
Multilevel Picard approximations overcome the curse of dimensionality when approximating semilinear heat equations with gradient-dependent nonlinearities in $L^p$-sense
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions
University of Münster（明斯特大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Shenzhen Research Institute of Big Data（深圳大数据研究院）
梯度下降（GD）型最优化方法是用整流线性单元（ReLU）激活来训练人工神经网络（ANN）的标准工具。尽管GD型最优化方法在用ReLU激活训练ANN的数值模拟中取得了巨大成功，它仍然是-即使在最简单的情况下，普通香草GD最优化方法与随机初始化和人工神经网络与一个隐藏层-一个开放的问题，以证明（或反驳）GD最优化方法的风险在这样的ANN的训练中收敛到零作为ANN的宽度的猜想，独立随机初始化的数量和GD步骤的数量增加到无穷大。在本文中，我们证明了这一猜想的情况下，输入数据的概率分布是一个紧凑的区间上的连续均匀分布，其中的概率分布的神经网络参数的随机初始化是标准正态分布，和所考虑的目标函数是连续和分段仿射线性。粗略地说，在我们的数学收敛分析中的关键成分是（i）证明风险函数的全局最小值的合适集合是\emph{twice continuously differentiable submanifolds of the ANN parameter spaces}，（ii）证明这些全局最小值的集合上的风险函数的Hessian满足适当的\emph{maximal rank condition}，并且，此后，（iii）应用[Fehrman，B.，Gess，B.，Jentzen，A.，非凸目标函数的随机梯度下降法的收敛速度。J·马赫。学习. Res. 21（136）：1- 48，2020]以建立具有随机初…
Multilevel Picard approximations and deep neural networks with ReLU, leaky ReLU, and softplus activation overcome the curse of dimensionality when approximating semilinear parabolic partial differential equations in $L^p$-sense
Division of Mathematical Sciences, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore（数学科学系，物理与数学科学学院，南洋理工大学，新加坡）；Faculty of Mathematics, Bielefeld University, Germany（数学学院，比勒菲尔德大学，德国）
