连续分段线性函数的稳定参数化
Stable Parametrization of Continuous and Piecewise-Linear Functions
摘要：
在深度学习中发挥重要作用的整流线性单元（ReLU）神经网络生成连续和分段线性（CPWL）函数。虽然它们提供了强大的参数表示，但参数空间和函数空间之间的映射缺乏稳定性。在本文中，我们研究了一种替代表示的CPWL功能，依赖于当地的帽子基函数。它基于这样一个事实，即任何CPWL函数都可以由三角测量及其在网格点的值指定。我们给出了帽函数在任意维数的三角剖分上形成Riesz基的充要条件，从而保证了参数与相应的CPWL函数之间的联系是稳定的和唯一的.此外，我们提供了一个估计的$\ell_2\rightarrow L_2$条件数的本地表示。最后，作为我们的框架的一个特殊情况下，我们专注于一个系统的参数化的$\mathbb{R}^d$与控制点放置在一个统一的网格。特别是，我们选择的帽子基函数是一个单一的线性盒样条的移位副本。在这种情况下，我们证明了我们的一般估计的条件数是最佳的。我们还将我们的局部表示与基于因果ReLU样函数的移位的非局部表示相关联。
英文摘要：
Rectified-linear-unit (ReLU) neural networks, which play a prominent role in deep learning, generate continuous and piecewise-linear (CPWL) functions. While they provide a powerful parametric representation, the mapping between the parameter and function spaces lacks stability. In this paper, we investigate an alternative representation of CPWL functions that relies on local hat basis functions. It is predicated on the fact that any CPWL function can be specified by a triangulation and its values at the grid points. We give the necessary and sufficient condition on the triangulation (in any number of dimensions) for the hat functions to form a Riesz basis, which ensures that the link between the parameters and the corresponding CPWL function is stable and unique. In addition, we provide an estimate of the $\ell_2\rightarrow L_2$ condition number of this local representation. Finally, as a special case of our framework, we focus on a systematic parametrization of $\mathbb{R}^d$ with control points placed on a uniform grid. In particular, we choose hat basis functions that are shifted replicas of a single linear box spline. In this setting, we prove that our general estimate of the condition number is optimal. We also relate our local representation to a nonlocal one based on shifts of a causal ReLU-like function.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
