白化和二阶优化都使数据集中的信息在训练期间不可用，并且可以减少或防止泛化
Whitening and second order optimization both make information in the dataset unusable during training, and can reduce or prevent generalization
摘要：
机器学习基于泛化的概念：在足够大的训练集上实现低误差的模型也应该在来自相同分布的新样本上表现良好。我们表明，数据白化和二阶优化可以损害或完全防止泛化。一般来说，模型训练利用数据集的样本-样本二阶矩矩阵中包含的信息。对于一般的一类模型，即模型的第一层是全连通的，我们证明了包含在这个矩阵中的信息是唯一的信息，可以用来推广。使用白化数据或某些二阶优化方案训练的模型对这些信息的访问较少，导致泛化能力降低或不存在。我们通过实验验证了这些预测的几个架构，并进一步证明，泛化继续受到损害，即使当理论要求放宽。然而，我们也通过实验证明了正则化二阶优化可以提供一个实际的权衡，即训练速度加快，但丢失的信息更少，在某些情况下，泛化甚至可以得到改善。
英文摘要：
Machine learning is predicated on the concept of generalization: a model achieving low error on a sufficiently large training set should also perform well on novel samples from the same distribution. We show that both data whitening and second order optimization can harm or entirely prevent generalization. In general, model training harnesses information contained in the sample-sample second moment matrix of a dataset. For a general class of models, namely models with a fully connected first layer, we prove that the information contained in this matrix is the only information which can be used to generalize. Models trained using whitened data, or with certain second order optimization schemes, have less access to this information, resulting in reduced or nonexistent generalization ability. We experimentally verify these predictions for several architectures, and further demonstrate that generalization continues to be harmed even when theoretical requirements are relaxed. However, we also show experimentally that regularized second order optimization can provide a practical tradeoff, where training is accelerated but less information is lost, and generalization can in some circumstances even improve.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
