AI 中文总结
本文针对良性插值现象的新解释展开哲学层面的辩论,指出其诉诸单个模型简单性的做法缺乏与泛化的可证关联,造成了解释缺口。
AI 中文摘要
当代深度学习方法即使在完美拟合训练数据时也能良好泛化,这一现象被称为良性插值。经典统计学习理论无法解释该现象,促使统计学与机器学习领域提出一系列新解释。这些新方案的共同特征是诉诸插值模型的简单性偏好,常表现为奥卡姆剃刀的一种形式。本文为哲学领域读者澄清了这场辩论,并认为这种对简单性的新诉诸造成了解释缺口。经典理论提供了将模型类的简单性与良好泛化关联起来的定理,从而支持方法论的简单性准则;而新解释则诉诸单个模型的属性,并将其解读为一种简单性。由于缺乏与泛化的可证关联,原本由定理承担的工作现在由“简单性”这一名称完成,使得一个实质性且未加论证的假设看起来像是对熟悉方法论原则的应用。
英文摘要
Contemporary deep learning methods generalize well even when they fit their training data perfectly, a phenomenon known as benign interpolation. This phenomenon cannot be accounted for by classical statistical learning theory and has prompted a range of attempted new explanations in the statistics and machine learning literature. A common feature of these new proposals is an appeal to a simplicity preference among interpolating models, often presented as a form of Occam's razor. We clarify this debate for a philosophical audience and argue that this new appeal to simplicity creates an explanatory gap. The classical theory offers theorems which connect the simplicity of model classes to good generalization, thus underwriting methodological simplicity norms. The new accounts instead appeal to properties of individual models, which they interpret as a kind of simplicity. Lacking a provable connection to generalization, it is the name "simplicity" that does the work a theorem used to do, making a substantive and unargued assumption look like the application of a familiar methodological principle.