通过缩放在软阈值和硬阈值之间桥接
Bridging between soft and hard thresholding by scaling
摘要：
在这篇文章中，我们开发和分析了一种阈值方法，其中软阈值估计独立扩展的经验尺度值。缩放值具有共同的超参数，该超参数是实现硬阈值化的理想缩放值的扩展阶。我们简单地称这个估计器为缩放软阈值估计器。比例软阈值法是一种通用的方法，它包括软阈值法和非负绞喉法的特殊情况，并给出了自适应LASSO的另一种推导。然后，我们通过Stein的无偏风险估计导出了缩放软阈值的自由度，并发现它被分解为软阈值的自由度和连接到硬阈值的剩余部分。在这个意义上，缩放的软阈值化给出了软阈值化方法和硬阈值化方法之间的自然桥梁。由于自由度表示过拟合的程度，因此该结果意味着在缩放的软阈值化中存在两个过拟合源。源自软阈值的第一源由未移除系数的数量确定，并且是过拟合程度的自然度量。我们分析了第二个源在一个特定的情况下的缩放软阈值通过参考硬阈值的已知结果。然后，我们发现，在一个稀疏的，大样本和非参数设置，第二个源很大程度上是由系数估计，其真实值为零，并有过拟合的影响时，阈值水平在这些系数估计的噪声水平。在一个简单的数值例子中，这些理论含义很好地解释了自由度的行为。同时，基于本文的结果和一些已知的事实，我们解释了软阈值、硬阈值和尺度软阈值方法的风险行为。
英文摘要：
In this article, we developed and analyzed a thresholding method in which soft thresholding estimators are independently expanded by empirical scaling values. The scaling values have a common hyper-parameter that is an order of expansion of an ideal scaling value that achieves hard thresholding. We simply call this estimator a scaled soft thresholding estimator. The scaled soft thresholding is a general method that includes the soft thresholding and non-negative garrote as special cases and gives an another derivation of adaptive LASSO. We then derived the degree of freedom of the scaled soft thresholding by means of the Stein's unbiased risk estimate and found that it is decomposed into the degree of freedom of soft thresholding and the reminder connecting to hard thresholding. In this meaning, the scaled soft thresholding gives a natural bridge between soft and hard thresholding methods. Since the degree of freedom represents the degree of over-fitting, this result implies that there are two sources of over-fitting in the scaled soft thresholding. The first source originated from soft thresholding is determined by the number of un-removed coefficients and is a natural measure of the degree of over-fitting. We analyzed the second source in a particular case of the scaled soft thresholding by referring a known result for hard thresholding. We then found that, in a sparse, large sample and non-parametric setting, the second source is largely determined by coefficient estimates whose true values are zeros and has an influence on over-fitting when threshold levels are around noise levels in those coefficient estimates. In a simple numerical example, these theoretical implications has well explained the behavior of the degree of freedom. Moreover, based on the results here and some known facts, we explained the behaviors of risks of soft, hard and scaled soft thresholding methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
