神经网络的平均维数--是什么导致了交互效应？
The Mean Dimension of Neural Networks -- What causes the interaction effects?
摘要：
Owen和Hoyt最近指出，有效维数提供了人工神经网络输入-输出映射的关键结构信息。沿着这条研究路线，这项工作提出了一个估计过程，允许从一个给定的数据集的平均尺寸的计算，而不从外部分布的restering。当特征是独立的时，设计产生总指数，当特征是相关的时，产生总指数的变体。我们证明了这个变体具有零独立性。通过合成数据集，我们分析了平均维度如何逐层演变，以及激活函数如何影响相互作用的大小。然后，我们使用平均维数来研究一些最广泛使用的图像识别卷积架构（LeNet，ResNet，DenseNet）。为了考虑像素相关性，我们建议在添加逆PCA层之后计算平均维度，该逆PCA层允许人们处理不相关的PCA变换特征，而无需重新训练神经网络。我们使用广义总指数来产生热图用于事后解释，并且我们采用PCA转换特征的平均维度来交叉比较人工神经网络结构。结果提供了一些关于跨架构的交互幅度差异的见解，以及关于平均维度在训练期间如何演变的指示。
英文摘要：
Owen and Hoyt recently showed that the effective dimension offers key structural information about the input-output mapping underlying an artificial neural network. Along this line of research, this work proposes an estimation procedure that allows the calculation of the mean dimension from a given dataset, without resampling from external distributions. The design yields total indices when features are independent and a variant of total indices when features are correlated. We show that this variant possesses the zero independence property. With synthetic datasets, we analyse how the mean dimension evolves layer by layer and how the activation function impacts the magnitude of interactions. We then use the mean dimension to study some of the most widely employed convolutional architectures for image recognition (LeNet, ResNet, DenseNet). To account for pixel correlations, we propose calculating the mean dimension after the addition of an inverse PCA layer that allows one to work on uncorrelated PCA-transformed features, without the need to retrain the neural network. We use the generalized total indices to produce heatmaps for post-hoc explanations, and we employ the mean dimension on the PCA-transformed features for cross comparisons of the artificial neural networks structures. Results provide several insights on the difference in magnitude of interactions across the architectures, as well as indications on how the mean dimension evolves during training.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
