电子配件
Fitting Elephants
摘要:
教科书上的智慧主张平滑的函数拟合,并暗示噪声数据的插值应该导致较差的泛化。一个相关的启发是拟合参数应该少于测量值(奥卡姆剃刀)。令人惊讶的是,当代机器学习(ML)方法,参见深度网络(DNN),尽管插值噪声数据,但泛化良好。这可以通过统计一致插值(SCI)来理解,即最佳地概括大数据的数据插值技术。在这篇文章中,我们阐明SCI使用加权插值最近邻(wiNN)算法,它增加了奇异权函数的kNN(k-最近邻)。这表明数据插值可以成为大数据的有效ML策略。SCI阐明了建模自然现象的两种方法之间的关系:理论物理的理性主义方法(强先验),参数很少,现代ML的非理性主义方法(弱先验),参数多于数据。SCI表明,纯粹的经验方法可以成功地预测。然而,数据插值不提供理论见解,并且训练数据要求可能是禁止的。复杂的动物大脑介于这两个极端之间,具有许多参数,但训练数据有限,并且具有编码在物种特异性中尺度电路中的先前结构。因此,现代ML提供了一种不同于物理理论和动物大脑的独特认识论方法。
英文摘要:
Textbook wisdom advocates for smooth function fits and implies that interpolation of noisy data should lead to poor generalization. A related heuristic is that fitting parameters should be fewer than measurements (Occam's Razor). Surprisingly, contemporary machine learning (ML) approaches, cf. deep nets (DNNs), generalize well despite interpolating noisy data. This may be understood via Statistically Consistent Interpolation (SCI), i.e. data interpolation techniques that generalize optimally for big data. In this article we elucidate SCI using the weighted interpolating nearest neighbors (wiNN) algorithm, which adds singular weight functions to kNN (k-nearest neighbors). This shows that data interpolation can be a valid ML strategy for big data. SCI clarifies the relation between two ways of modeling natural phenomena: the rationalist approach (strong priors) of theoretical physics with few parameters and the empiricist (weak priors) approach of modern ML with more parameters than data. SCI shows that the purely empirical approach can successfully predict. However data interpolation does not provide theoretical insights, and the training data requirements may be prohibitive. Complex animal brains are between these extremes, with many parameters, but modest training data, and with prior structure encoded in species-specific mesoscale circuitry. Thus, modern ML provides a distinct epistemological approach different both from physical theories and animal brains.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。