ANOVA平均维数的有效估计及其在神经网络分类中的应用
Efficient estimation of the ANOVA mean dimension, with an application to neural net classification
摘要：
$d$变量的黑盒函数的平均维数是一种方便的方法来总结它受高阶或低阶相互作用支配的程度。它用$2^d-1$方差分量表示，但也可以写成$d$ Sobol指数之和，可以用留一法估计。我们比较了这些留一法的方差：吉布斯采样器称为蜿蜒的楼梯，径向采样器，改变每个变量一次从基线，和一个天真的采样器，从来没有重复使用函数的评估，所以成本约为其他方法的两倍。对于加法函数，径向和缠绕阶梯是最有效的。对于乘法函数，如果因子具有高峰度，则朴素方法可以容易地是最有效的。作为一个例子，我们认为平均尺寸的神经网络分类器的数字从MNIST数据集。分类器是$784$像素的函数。对于这个问题，螺旋楼梯是最好的算法。我们发现，最终softmax层的输入具有从$1.35$到$2.0$的平均维度。
英文摘要：
The mean dimension of a black box function of $d$ variables is a convenient way to summarize the extent to which it is dominated by high or low order interactions. It is expressed in terms of $2^d-1$ variance components but it can be written as the sum of $d$ Sobol' indices that can be estimated by leave one out methods. We compare the variance of these leave one out methods: a Gibbs sampler called winding stairs, a radial sampler that changes each variable one at a time from a baseline, and a naive sampler that never reuses function evaluations and so costs about double the other methods. For an additive function the radial and winding stairs are most efficient. For a multiplicative function the naive method can easily be most efficient if the factors have high kurtosis. As an illustration we consider the mean dimension of a neural network classifier of digits from the MNIST data set. The classifier is a function of $784$ pixels. For that problem, winding stairs is the best algorithm. We find that inputs to the final softmax layer have mean dimensions ranging from $1.35$ to $2.0$.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
