瑞士军队的无限小折刀
A Swiss Army Infinitesimal Jackknife
摘要：
机器学习算法的误差或可变性通常通过使用观察数据的不同加权版本重复重新拟合模型来评估。普遍存在的交叉验证（CV）工具和bootstrap就是这种技术的例子。这些方法在很大程度上是强大的，因为它们的模型不可知论，但由于需要反复重新拟合模型，因此在现代大型数据集上运行可能很慢。在这项工作中，我们使用线性近似的拟合过程的权重的依赖性，产生的结果，可以比重复重新拟合的数量级更快。这种线性近似在统计学文献中有时被称为“无穷小折刀”，它主要用作证明渐近结果的理论工具。我们提供了明确的有限样本误差界的无穷小折刀在一个简单的，可验证的假设。我们的研究结果适用的权重和数据是否是随机的或确定性的，因此可以被用来作为一种工具，证明各种各样的问题的无穷小折刀的准确性。作为一个推论，我们状态温和的正则条件下，我们的近似一致估计真正的离开- $k$ -出交叉验证任何固定的$k$。这些理论结果与现代自动微分软件一起，支持无穷小折刀应用于机器学习中的各种实际问题，提供了“瑞士军队无穷小折刀”。我们证明了我们的方法在一系列模拟和真实数据集上的准确性。
英文摘要：
The error or variability of machine learning algorithms is often assessed by repeatedly re-fitting a model with different weighted versions of the observed data. The ubiquitous tools of cross-validation (CV) and the bootstrap are examples of this technique. These methods are powerful in large part due to their model agnosticism but can be slow to run on modern, large data sets due to the need to repeatedly re-fit the model. In this work, we use a linear approximation to the dependence of the fitting procedure on the weights, producing results that can be faster than repeated re-fitting by an order of magnitude. This linear approximation is sometimes known as the "infinitesimal jackknife" in the statistics literature, where it is mostly used as a theoretical tool to prove asymptotic results. We provide explicit finite-sample error bounds for the infinitesimal jackknife in terms of a small number of simple, verifiable assumptions. Our results apply whether the weights and data are stochastic or deterministic, and so can be used as a tool for proving the accuracy of the infinitesimal jackknife on a wide variety of problems. As a corollary, we state mild regularity conditions under which our approximation consistently estimates true leave-$k$-out cross-validation for any fixed $k$. These theoretical results, together with modern automatic differentiation software, support the application of the infinitesimal jackknife to a wide variety of practical problems in machine learning, providing a "Swiss Army infinitesimal jackknife". We demonstrate the accuracy of our methods on a range of simulated and real datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
