编码理论与交叉验证之间的联系及其应用
A Link between Coding Theory and Cross-Validation with Applications
AI总结:
本文建立编码理论与交叉验证的联系,证明零或非零LPOCV误差下可解分类问题最大数等于常重量码或轻量常重量码的码字数,并给出界及推广的随机化检验。
摘要:
一个单一的学习算法在固定数据上能够解决多少个不同的二分类问题,且交叉验证误差恰好为零或至多给定数量?虽然在前一种情况下,该数量已知受限于没有免费午餐定理,但我们表明,确切的答案由检错编码理论给出。作为案例研究,我们聚焦于AUC性能指标和留一对交叉验证(LPOCV),其中每次留出所有具有不同类别标签的数据对中的一对。我们表明,在固定类别比例下,学习算法能够实现零LPOCV误差的分类问题的最大数量,等于具有特定技术性质的常重量码(CWC)中码字的最大数量。然后,我们通过引入轻量常重量码(light CWCs)来推广常重量码,并证明了非零LPOCV误差与轻量常重量码之间的类似结果。此外,我们证明了轻量常重量码中码字最大数量的上界和下界。最后,作为一个直接的实际应用,我们开发了基于LPOCV的新的学习算法随机化检验,该检验推广了经典的Wilcoxon-Mann-Whitney U检验。
英文摘要:
How many different binary classification problems a single learning algorithm can solve on a fixed data with exactly zero or at most a given number of cross-validation errors? While the number in the former case is known to be limited by the no-free-lunch theorem, we show that the exact answers are given by the theory of error detecting codes. As a case study, we focus on the AUC performance measure and leave-pair-out cross-validation (LPOCV), in which every possible pair of data with different class labels is held out at a time. We show that the maximal number of classification problems with fixed class proportion, for which a learning algorithm can achieve zero LPOCV error, equals the maximal number of code words in a constant weight code (CWC), with certain technical properties. We then generalize CWCs by introducing light CWCs, and prove an analogous result for nonzero LPOCV errors and light CWCs. Moreover, we prove both upper and lower bounds on the maximal numbers of code words in light CWCs. Finally, as an immediate practical application, we develop new LPOCV based randomization tests for learning algorithms that generalize the classical Wilcoxon-Mann-Whitney U test.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。