基于Hessian-Schatten全变分的学习方案复杂性度量
Measuring Complexity of Learning Schemes Using Hessian-Schatten Total Variation
摘要:
在本文中,我们介绍了Hessian-Schatten全变差(HTV)-一种新的方法,量化的总“粗糙度”的多元函数。我们定义HTV的动机是评估监督学习计划的复杂性。我们首先指定适当的矩阵值Banach空间,配备了适当的类的混合规范。然后,我们表明,HTV是不变的旋转,缩放和平移。此外,它的最小值是针对线性映射实现的,这支持了线性回归是最不复杂的学习模型的普遍直觉。接下来,我们提出了两个一般类的函数的HTV的封闭形式的表达式。第一个是具有一定规律性的Sobolev函数类,我们证明了HTV与Hessian-Schatten正则化函数相一致,Hessian-Schatten正则化函数有时被用作图像重建的正则化函数。第二类是连续分段线性(CPWL)函数类。在这种情况下,我们表明,HTV反映了线性区域之间的斜率有一个共同的小平面的总变化。因此,它可以被看作是CPWL映射的线性区域数(l0-型)的凸松弛(l1-型)。最后,我们说明了我们提出的算法的使用。
英文摘要:
In this paper, we introduce the Hessian-Schatten total variation (HTV) -- a novel seminorm that quantifies the total "rugosity" of multivariate functions. Our motivation for defining HTV is to assess the complexity of supervised-learning schemes. We start by specifying the adequate matrix-valued Banach spaces that are equipped with suitable classes of mixed norms. We then show that the HTV is invariant to rotations, scalings, and translations. Additionally, its minimum value is achieved for linear mappings, which supports the common intuition that linear regression is the least complex learning model. Next, we present closed-form expressions of the HTV for two general classes of functions. The first one is the class of Sobolev functions with a certain degree of regularity, for which we show that the HTV coincides with the Hessian-Schatten seminorm that is sometimes used as a regularizer for image reconstruction. The second one is the class of continuous and piecewise-linear (CPWL) functions. In this case, we show that the HTV reflects the total change in slopes between linear regions that have a common facet. Hence, it can be viewed as a convex relaxation (l1-type) of the number of linear regions (l0-type) of CPWL mappings. Finally, we illustrate the use of our proposed seminorm.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。