乳腺癌转移深度FNN预测模型过拟合的实证研究
Empirical Study of Overfitting in Deep FNN Prediction Models for Breast Cancer Metastasis
摘要:
过度拟合是指当前模型与特定数据集完美拟合,导致泛化能力减弱,最终可能影响预测未来数据的准确性。在这项研究中,我们使用有关乳腺癌转移的EHR数据集来研究深度前馈神经网络(FNN)预测模型的过拟合。我们包含了深度FNN模型的11个超参数,并采取了一种实证方法来研究这些超参数中的每一个在给定大范围值时如何影响预测性能和过拟合。我们还研究了一些有趣的超参数对如何相互作用以影响模型性能和过拟合。我们研究的11个超参数包括激活函数、权重初始化器、隐藏层数、学习率、动量、衰减、辍学率、批量大小、历元、L1和L2。我们的研究结果表明,大多数单超参数与模型预测性能和过拟合呈负或正相关。特别是,我们发现过度拟合总体上倾向于与学习率、衰减、批次侧和L2负相关,但倾向于与动量、时期和L1正相关。根据我们的研究结果,学习率、衰减和批量大小可能对过拟合和预测性能的影响比大多数其他超参数(包括L1、L2和辍学率)更显著,这些参数是为了最小化过拟合而设计的。我们还发现了一些有趣的相互作用的超参数对,如学习率和动量,学习率和衰减,批量大小和历元。关键词:深度学习,过拟合,预测,网格搜索,前馈神经网络,乳腺癌转移。
英文摘要:
Overfitting is defined as the fact that the current model fits a specific data set perfectly, resulting in weakened generalization, and ultimately may affect the accuracy in predicting future data. In this research we used an EHR dataset concerning breast cancer metastasis to study overfitting of deep feedforward Neural Networks (FNNs) prediction models. We included 11 hyperparameters of the deep FNNs models and took an empirical approach to study how each of these hyperparameters was affecting both the prediction performance and overfitting when given a large range of values. We also studied how some of the interesting pairs of hyperparameters were interacting to influence the model performance and overfitting. The 11 hyperparameters we studied include activate function; weight initializer, number of hidden layers, learning rate, momentum, decay, dropout rate, batch size, epochs, L1, and L2. Our results show that most of the single hyperparameters are either negatively or positively corrected with model prediction performance and overfitting. In particular, we found that overfitting overall tends to negatively correlate with learning rate, decay, batch sides, and L2, but tends to positively correlate with momentum, epochs, and L1. According to our results, learning rate, decay, and batch size may have a more significant impact on both overfitting and prediction performance than most of the other hyperparameters, including L1, L2, and dropout rate, which were designed for minimizing overfitting. We also find some interesting interacting pairs of hyperparameters such as learning rate and momentum, learning rate and decay, and batch size and epochs. Keywords: Deep learning, overfitting, prediction, grid search, feedforward neural networks, breast cancer metastasis.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。