几何和泛化:特征值作为网络泛化失败的预测因子
Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalize
摘要:
我们通过训练的权重矩阵的雅可比矩阵研究了训练的自编码器对输入空间的变形。在这样做时,我们证明了输入空间中的点的均方误差的界限,关于特征向量的正交性的假设下。我们还表明,迹和积的特征值的雅可比矩阵是一个很好的预测的MSE测试点。这是一种独立于数据集的方法,用于测试自动编码器对新输入进行泛化的能力。也就是说,不需要训练网络的数据集的知识,只需要训练模型的参数。
英文摘要:
We study the deformation of the input space by a trained autoencoder via the Jacobians of the trained weight matrices. In doing so, we prove bounds for the mean squared errors for points in the input space, under assumptions regarding the orthogonality of the eigenvectors. We also show that the trace and the product of the eigenvalues of the Jacobian matrices is a good predictor of the MSE on test points. This is a dataset independent means of testing an autoencoder's ability to generalize on new input. Namely, no knowledge of the dataset on which the network was trained is needed, only the parameters of the trained model.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。