SOAK：用于估计数据子集中模式相似性的Same/Other/All K折交叉验证
SOAK: Same/Other/All K-fold cross-validation for estimating similarity of patterns in data subsets
AI总结：
提出SOAK交叉验证方法，通过系统比较不同数据子集上训练的模型在固定测试集上的预测表现，估计子集间可学习模式的相似性，以判断数据合并的益处及新子集预测的准确性。
摘要：
在机器学习的许多实际应用中，我们想知道是否能够使用目前已收集的数据进行训练，并在某些方面（时间段、地理区域等）存在定性差异的新测试数据子集上获得准确的预测。另一个问题是，数据子集是否足够相似，以至于在模型训练期间合并子集是有益的。我们提出了SOAK（Same/Other/All K折交叉验证），这是一种可用于回答这两个问题的新方法。SOAK系统地比较在不同数据子集上训练并用于在固定测试子集上进行预测的模型，以估计数据子集中可学习/可预测模式的相似性。我们展示了在六个新真实数据集（包含地理/时间子集，以检查预测在新子集上是否准确）、3个图像对数据集（子集是不同的图像类型，以检查在相似图像上是否获得更小的预测误差）和11个具有预定义训练/测试划分的基准数据集（以检查预定义划分的相似性）上使用SOAK的结果。
英文摘要：
In many real-world applications of machine learning, we are interested to know if it is possible to train on the data that we have gathered so far, and obtain accurate predictions on a new test data subset that is qualitatively different in some respect (time period, geographic region, etc). Another question is whether data subsets are similar enough so that it is beneficial to combine subsets during model training. We propose SOAK, Same/Other/All K-fold cross-validation, a new method which can be used to answer both questions. SOAK systematically compares models which are trained on different subsets of data, and then used for prediction on a fixed test subset, to estimate the similarity of learnable/predictable patterns in data subsets. We show results of using SOAK on six new real data sets (with geographic/temporal subsets, to check if predictions are accurate on new subsets), 3 image pair data sets (subsets are different image types, to check that we get smaller prediction error on similar images), and 11 benchmark data sets with predefined train/test splits (to check similarity of predefined splits).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
