高维空间中的交叉验证:最小二乘模型和多类LDA的生命线
Cross-validation in high-dimensional spaces: a lifeline for least-squares models and multi-class LDA
摘要:
最小二乘模型,如线性回归和线性判别分析(LDA)是最流行的统计学习技术之一。然而,由于它们的计算时间随着特征数量的增加而立方地增加,因此它们在高维神经成像数据集中效率低下。幸运的是,对于k折交叉验证,已经开发了一种分析方法,可以在最小二乘模型中产生精确的交叉验证预测,而无需显式训练模型。它的计算时间随着测试样本的数量而增长。在这里,我们在交叉验证和排列测试的背景下系统地研究了这种方法。LDA是用于示范,但所有其他最小二乘方法的结果。此外,一个非平凡的扩展,多类LDA正式推导。使用复杂性计算,模拟和置换测试的EEG/MEG数据集的分析方法进行评估。根据特征和样本之间的比例,分析方法比标准方法(在每个训练集上重新训练模型)快10,000倍。这允许在高维数据中对最小二乘模型和多类LDA进行快速交叉验证,在多维数据集,代表性相似性分析和排列测试中具有明显的应用。
英文摘要:
Least-squares models such as linear regression and Linear Discriminant Analysis (LDA) are amongst the most popular statistical learning techniques. However, since their computation time increases cubically with the number of features, they are inefficient in high-dimensional neuroimaging datasets. Fortunately, for k-fold cross-validation, an analytical approach has been developed that yields the exact cross-validated predictions in least-squares models without explicitly training the model. Its computation time grows with the number of test samples. Here, this approach is systematically investigated in the context of cross-validation and permutation testing. LDA is used exemplarily but results hold for all other least-squares methods. Furthermore, a non-trivial extension to multi-class LDA is formally derived. The analytical approach is evaluated using complexity calculations, simulations, and permutation testing of an EEG/MEG dataset. Depending on the ratio between features and samples, the analytical approach is up to 10,000x faster than the standard approach (retraining the model on each training set). This allows for a fast cross-validation of least-squares models and multi-class LDA in high-dimensional data, with obvious applications in multi-dimensional datasets, Representational Similarity Analysis, and permutation testing.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。